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FOREWORD 


One  of  the  research  goals  of  the  Decision  Sciences  Laboratory 
is  the  development  of  design  principles  for  automated  training  subsystems 
which  could  be  built-into  future  Information  Systems.  Such  subsystems 
would  provide  Information  Systems  with  the  capability  of  training 
automatically  their  own  operators.  The  need  for  such  on-the-job 
training  has  already  become  apparent.  To  be  able  to  design  such  a 
capability  requires  first  the  solution  of  many  conceptual  and  experimental 
problems . 

Task  768204,  Automated  Training  for  Information  Systems,  under 
Project  7682,  Man-Computer  Information  Processing,  was  established 
to  formulate  and  answer  some  of  these  problems. 

This  report  is  one  in  a  planned  series  supporting  Task  768204.  It 
was  prepared  as  a  joint  effort  of  the  staff  of  Bio-Dynamics,  Inc.  , 
working  under  terms  of  Contract  AF  19(628) -455,  and  Decision 
Sciences  Laboratory,  Electronic  Systems  Division.  The  authors 
are  Dr.  Thomas  B.  Sheridan,  Principal  Investigator  and  Dr.  Sylvia  R. 
Mayer,  Contract  Monitor. 

Several  Bio -Dynamics  staff  members  are  contributing  directly  in 
the  accomplishment  of  different  parts  of  this  study:  Ms.  S.  Molnar, 

R.  Rosenberg  and  J.  Mickunas  on  the  work  described  in  Chapter  III, 

Dr.  B.  Duggar:  Chapter  IV,  Dr.  A.  Johnson:  Chapter  V. 
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DESIGN  AND  USE  OF  INFORMATION  SYSTEMS 


FOR  AUTOMATED  ON-THE-JOB  TRAINING 


I.  Conceptual  and  Experimental  Approaches 


Abstract 


This  report  describes  exploratory  developments  on  laboratory  models 
of  automated  training  subsystems  for  Information  Systems.  Such  subsystems 
could  provide  future  Information  Systems  with  the  capability  to  train  their 
users  on-the-job. 

The  report  outlines  on-going  studies  concerned  with  (1)  the  unique  train¬ 
ing  requirements  in  advanced  Information  Systems;  (2)  new  training  concepts 
and  techniques  to  meet  these  requirements;  and  (3)  an  analytic  tool  to  des¬ 
cribe  functional  and  structural  overlap  of  equipment  required  for  both  opera¬ 
tions  and  training. 
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Introduction 


Air  Force  Information  Systems  perform  many  complex  "intellectual  M 
functions  which,  until  recently,  have  been  regarded  as  the  exclusive 
province  of  man.  The  research  reported  herein  is  based  on  the  premise 
that,  with  a  modicum  of  new  design  principles  and  human  engineering, 
these  Information  Systems  could  do  much  more:  they  could  also  train 
their  human  components  to  use  and  maintain  them.  This  report  describes 
initial  research  efforts  aimed  at  developing  these  automated*  training 
subsystems  for  Information  Systems. 

A.  Automated  Training  Research  in  the  Air  Force:  Past  and  Present 

Air  Force  research  and  development  in  automated  training  has  been 
in  progress  for  more  than  a  decade.  Techniques,  devices  and  theory  for 
automated  training  have  evolved  gradually.  At  the  present  time  research 
on  various  aspects  of  automated  training  is  being  carried  on  by  Air  Force 
Systems  Command  (2,  14,  15,  17),  and  Air  Training  Command  is  conducting 
an  intensive  program  of  experimental  applications  of  these  techniques  (16). 
Completed  studies  have  left  little  doubt  as  to  the  potential  value  of  this 
approach  for  the  Air  Force  (6,  8,  9,  11,  12,  27).  Questions  of  feasibility 
and  suitability  are  no  longer  posed.  Rather  the  current  problem  is  one  of 
improving  and  refining  primitive  methods  and  concepts  for  more  efficient 
implementation  in  a  variety  of  Air  Force  training  operations. 

B.  Automated  Training  Research  in  the  Air  Force:  Future 

The  majority  of  studies  have  been  concerned  with  part-task  training 
in  classroom  settings.  This  emphasis  has  been  proportioned  to  the  past 

*  As  used  here  this  term  includes  the  methods  and  concepts  of  programed 
instruction  plus  machine  control  and  direction  of  that  instruction. 
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and  current  practice  of  conducting  the  major  part  of  Air  Force  training 
in  formal  classroom  settings  prior  to  on-site  assignment.  Automated 
training  for  on-the-job  use  has  received  somewhat  less  research  emphasis. 
However,  many  characteristics  of  future  Air  Force  operations  point  to 
an  inevitable  future  shift  in  training  location.  The  recent  Path  Finder 
Study  (3)  initiated  by  L»t  Gen  James  E.  Briggs  of  Air  Training  Command, 
predicted  that  a  large  portion  of  future  training  must  occur  on-the-job, 
on-the -operational  equipment.  This  change  will  stem  from  the.  complex 
and  costly  nature  of  many  of  the  future  Air  Force  systems  -  particularly 
Information  Systems.  Classroom  simulators  of  the  traditional  type  will 
be  economically  unfeasible.  Factors  of  cost  in  dollars  and/or  cost  in 
floor  space  may  preclude  the  traditional  approach. 

Complex  simulators  will  probably  not  be  feasible  except  where 
danger  to  man,  machine  or  mission  prohibits  the  novices*  interaction  with 
operational  equipment.  The  main  advantage  of  conducting  training  at  the 
job  location  will  be  the  availability  of  operational  equipment  and  operational 
problems  for  MuseM  in  training.  Exactly  what  form  this  "use"  should  take 
or  how  it  should  be  accomplished  is  now  a  question  for  research. 

A  study  program  -  its  early  phases  are  described  herein  -  is 
researching  this  question  specifically  for  future  Air  Force  Information 
Systems . 

C.  Automated  Training  by  Information  Systems 
*  — 

This  approach  is  based  on  evidence  that  the  operational  equipment 
of  existing  and  future  Information  Systems  could  provide  the  physical 
requirements  for  an  automated  training  and  skill  maintenance  function 

^  Past  studies  in  the  SAGE  System  and  current  research  in  System  473L 
support  this  view. 
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without  interfering  with  the  primary  mission  function.  Furthermore, 
several  applications  of  automated  training  in  Information  Systems  have 
already  demonstrated  the  potential  of  this  method  for  the  training  tasks 
in  these  systems  (8,  9,  10,  11,  18,  27). 

The  long-range  objective  of  this  research  is  to  provide  basic 
principles  upon  which  to  design  into  an  otherwise  conventional  Information 
System  this  automated  training  function.  The  immediate  objective  is  to 
analyze  and  demonstrate  with  laboratory  models  how  the  logical  functions 
and  mechanisms  of  Information  System  tasks  and  associated  equipment 
overlap  with  the  functions  and  mechanisms  required  for  training  of  these  tasks. 

The  immediate  objective  is  being  accomplished  in  a  three -part 

plan: 

Phase  I.  This  phase  involves  modeling  of  artificial,  representative 
Information  System  tasks  first  in  graphical  symbology  and  then  in  hardware. 

This  phase  has  required  the  development  of  a  new  symbolic  language  which 
can  describe  both  man  and  machine  response  in  terms  easily  translatable 
into  electrical  circuitry.  Completed  exploratory  studies  are  described 
in  this  report,  Chapter  III. 

Phase  2.  This  phase  is  addressed  to  development  of  automated 
training  programs  tailored  to  Information  System  tasks.  These  automated 
training  programs  will  use  Phase  1  equipments  for  input-output  and  storage 
functions.  This  phase  is  still  in  its  formative  stages.  Completed  work, 
experimental  and  theoretical,  is  described  in  Chapters  IV  and  V. 

Phase  3.  In  this  phase  equipment  which  models  the  operational 
tasks  (Phase  1)  is  to  be  integrated  with  equipment  which  provides  the 
T - ; - 

This  interim  report  attempts  only  to  summarize  problem  formulations  and 
associated  exploratory  design  studies.  Individual  reports  will  be  issued  on  later 
formal  studies  with  scope  and  results  meriting  more  complete  documentation. 
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automated  training  functions  (Phase  2).  From  this  process  principles 
will  be  inferred  by  which  to  design  the  built-in  training  function.  This 
work  will  be  presented  in  a  later  volume  in  this  report  series. 
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II.  Summary 


The  long  range  objective  of  this  work  is  to  provide  principles  upon 
which  to  design  into  conventional  Information  Systems  the  capacity  to 
train  automatically  their  own  users.  Development  of  these  design 
principles  will  depend  on  research  findings  relative  to: 

1}  what  can  and  should  be  taught  to  enable  operators  to  utilize 
most  efficiently  Information  System  capabilities, 

2)  automated  training  procedures  for  Information  System  tasks, 

3)  degree  of  overlap  between  the  functions  and  structures  of 
the  normal  operating  task  and  equipment  and  the  functions 
and  structures  of  the  training  task  and  equipment. 

This  report  outlines  necessary  research,  describes  the  design  of 
studies  underway,  and  outlines  some  results  and  conclusions  derived  from 
completed  work.  Specific  recommendations  to  implement  findings  will 
appear  in  later  reports  in  this  series. 

The  experimental  and  theoretical  approaches  have  proceeded  along 
the  following  three  distinct  but  related- avenues: 

1)  a  Representative11  Air  Force  Information  System  task  was 

modeled  in  a  new  symbolic  language,  and  experiments  undertaken 
to  explore  methods  of  teaching  the  logic  flow  of  the  task, 

2}  two  ,!logicM  games  involving  querying-reasoning  behavior  are 
being  used  as  the  framework  for  the  study  of  such  behavior, 
and  evaluation  of  methods  of  teaching  optimum  strategy, 

3}  a  novel  concept  for  programed  learning  of  the  functions  of 
systems  composed  of  interrelated,  multiple  subsystems  has 
been  developed  and  experimental  validation  is  underway.  This 
concept  involves  the  nmethod  of  discovery"  and  the  "phylogenic" 
ordering  of  functions. 

Progress  along  each  of  these  avenues  is  described  in  separate  sections 
of  this  report. 
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HI.  Studies  on  a  Method  for  Self-Instruction  in  Task  Logic 


A.  Introduction 

A  sub -objective  of  the  overall  program  was  to  determine  and 
demonstrate,  through  use  of  experimental  task  or  model,  how  the  functions 
and  mechanisms  of  a  normal  operating  task  and  associated  equipment  may 
overlap  with  the  functions  and  mechanisms  of  a  training  program  and  equip¬ 
ment.  A  three  step  program  directed  towards  achieving  this  sub -objective 
was  defined  as  follows: 

1.  Development  of  a  narrative  and  hardware  model  of  a  representa¬ 
tive  Air  Force  Command  and  Control  System  Task. 

2.  Development  of  a  program  to  teach  this  task  to  a  novice  operator 
using  the  task  simulator  for  input -output  and  storage  function. 

3.  Integration  of  equipment  which  models  the  operational  task  with 
equipment  which  serves  to  teach  the  task  operation.  From  this, 
presumably,  principles  could  be  inferred  to  design  operational 

equipment  having  the  programed  operator  training  function  "built - 

•  n 
m  . 

The  first  step  has  been  completed,  the  second  and  third  have  been  com¬ 
pleted  for  several  alternative  teaching  configurations,  and  additional  experi¬ 
mental  evaluations  are  underway. 

The  "representative"  task  was  based  upon  the  activity  of  the 
SAGE  Intercept  Director  whose  job  it  is  to  follow  and  assist  an 
aircraft(s)  through  an  entire  mission.  This  SAGE  operation  was 
taken  as  exemplary  of  an  Air  Force  Command  and  Control  func¬ 
tion,  and  the  Intercept  Director  task  was  considered  a  workable  com¬ 
promise  between  command,  or  decision,  functions  and  rote  functions. 
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The  job  of  the  SAGE  Intercept  Director  (and  his  relationship  to  the  SAGE 
Intercept  Technician)  requires  a  decision  process,  leads  to  a  definite  outcome, 
and  can  be  subjected  to  a  procedural  analysis.  The  tasks  of  the  Intercept 
Director  and  the  Intercept  Technician  were  considered  as  one  task  since  it 
appeared  that  the  Director  would  have  to  know  the  Technician's  task  in  order  to 
perform  his  own  mission. 

Familiarization  with  the  task  was  gained  through  available  reports  (19  -24) 
as  well  as  a  cardboard  mock-up  and  a  prototype  instruction  program  designed 
by  the  American  Institute  of  Research.  *  The  entire  task  consists  of  "minor"  and 
"major"  sequential  elements.  Minor  sequences  are  the  smallest  independent 
actions  or  elements  of  the  overall  task.  A  minor  task  element  might  consist  of 
contact  with  the  pilot,  whereas  a  major  task  might  involve  following  an  aircraft 
throughout  its  mission.  Thus,  the  major  sequences  consist  of  blocks  of  minor 
sequences.  The  order  of  major  sequences  depends  upon  the  activities  of  the 
aircraft  and  the  priorities  of  the  situation.  There  is  no  fixed  number  of  minor 
sequences  in  a  major  sequence.  The  Intercept  Director  may  follow  an  assigned 
aircraft  throughout  its  mission,  or  he  may  terminate  his  assignment  if  the 
conditions  warrant.  (This  is  in  contrast  to  a  minor  sequence  where  once  the 
sequence  is  initiated  it  has  to  be  followed  through  until  the  correct  solution  is 
obtained. )  However,  once  a  specific  stimulus  appears,  a  corresponding  set  of 
responses  are  required  to  arrive  at  a  correct  solution. 

1  This  work,  conducted  under  contract  AF  19(604) -5951 ,  provided  the  first 
exploratory  development  on  automated,  on-the-job  training  for  Information 
Systems. 
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B.  Graphical  Model  of  the  Task 


A  task  analysis  and  graphical  description  of  the  logical  relation¬ 
ships  between  stimuli  and  correct  responses,  including  specification  of 
all  stimulus -response  contingencies  leading  to  correct  outcomes,  was 
performed.  For  simplification  of  the  procedural  analysis  of  the  Intercept 
Director's  task,  and  facilitation  of  its  modeling,  only  the  correct  actions 
were  included  in  the  analysis  and  graphical  representation.  In  addition, 
steps  which  were  overly  redundant  insofar  as  they  did  not  characterize 
any  "new"  logical  relationship  or  situation,  the  initial  phase  of  operation 
during  which  the  Intercept  Director  and  his  Technician  check  out  their 
equipment,  and  much  of  the  symbology  which  appears  on  the  Digital  Dis¬ 
play  screen  have  not  been  incorporated  in  the  present  model.  However, 
a  short  sequence  of  the  tracking  mission  illustrates  the  possibilities  of 
representing  the  logic  and  sequencing  of  SAGE  type  tasks  in  symbolic  form. 

A  limited  review  of  the  literature  in  analytic  symbology  suggested  no 
generally  accepted  symbols,  so  special  graphical  symbology  was  devised 
here  to  represent  individual  stimulus  responses  and  elements  of  the  task 
in  their  temporal  and  logical  relationships.  It  is  hoped  that  these  symbols 
will  be  of  use  to  other  research  of  a  similar  nature.  The  graphical  sym¬ 
bology  was  devised  with  general  objectives  in  mind: 
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1)  That  it  specify  unambiguously  and  without  redundancy  the  essential 
logical  relationships  in  the  tasks  under  consideration  (e.  g. ,  the  SAGE 
Intercept  Director  Task) 

2)  That  it  be  capable  of  specifying  unambiguously  and  without  redundancy 
the  logical  relationships  in  programed  training 

3)  That  the  format  of  presentation  have  the  time  or  sequential  continuum 
intuitively  apparent 

4)  That  the  symbols  and  their  format  be  able  to  be  drawn  freehand  as  the 
task  analysis  proceeds,  without  need  to  erase  or  redraw  (assuming  a 
modicum  of  planning  had  gone  into  choosing  a  large  enough  piece  of 
paper  or  scale  of  drawing) 

5)  That  the  symbolic  representation  be  easily  translatable  into  electrical 
circuitry. 

The  basic  symbols  used  to  describe  a  minor  element  are: 


O 


man 


□ 


control 


display 


The  direction  of  causality  is  invariably: 

man - »■  control,  control - display,  display - 


man. 


Logical  relationships  are  indicated  by  straight  lines  interconnecting  the 
elements  in  the  manner  of  a  signal -flow  graph.  Causation  is  always  from  left 
right,  i.  e.  an  input  is  on  the  left  side  of  an  element,  an  output  on  the  right.  For 
example,  an  action  element  depicted  by  the  following  picture 

0 — 0-0 - E 


can  be  described  narratively  as:  display  1  prompts  the  man  to  actuate  control  A. 
This  response  causes  display  2  to  appear,  signifying  the  correct  response  and 
the  beginning  of  a  new  task.  A  complete  catalog  of  the  symbols  used  to  specify 
task  elements  and  the  relationships  between  these  elements  appears  in  Appendix  I. 
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A  variety  of  branchings  may  be  required  to  indicate  the  logical 


relationships  of  elements  in  a  complex  task.  Those  used  in  the  graphical 


description  of  the  Intercept  Director's  task  are  described  below. 


1)  A  side  branch  indicates  that  the  straight  path  (action 
A)  is  taken  first,  then  the  closest  back  track  on  the 
side  path  (B),  then  the  next  closest  back  track  (C), 
and  so  on.  This  is  a  logical  "and",  with  ordered 
sequence. 

2)  An  unordered  "and "branch  indicates  that  A,  B,  and 
C  must  all  be  accomplished,  but  in  any  order. 

3)  A  decision  branch  indicates  that  the  action  to  be 
selected  is  contingent  upon  a  previous  branch  deci¬ 
sion  or  display,  as  indicated.  In  the  example  at  the 
left  A  is  selected  if  display  1;  B  if  display  2;  C  if 
display  3  led  to  this  action/ 

4)  A  "whimsical"  branch  is  indicated  by  a  closed  arrow. 
In  the  example  A,  or  B,  or  C  are  equally  satisfactory 
alternatives. 


The  decision  and 
ways.  For  example, 
path  has  been  taken  a 

made  among  A,  B,  C 


whimsical  branching  may  be  combined  in  several 


indicates  that  if  the  upper 
whimsical  choice  may  be 


and  D;  however,  if  display  1  appears  then  A  is 


chosen;  if  display  2,  then  B;  if  3,  then  C  (note  that  prior  to  this  step 
either  2  or  3  called  for  the  same  response).  Various  other  versions 
should  be  clear  from  the  context. 
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All  of  the  forms  of  branching  described  above  are  divergent  and 
indicate  actions  initiated  by  the  man.  Branching  may  also  be  convergent, 
between  control  and  display  or  between  display  and  man.  Thus,  the  dia¬ 
gram  on  the  left  indicates  display  C  is  operated  by  control  actions  A  and 


B,  whereas  the  diagram  on  the  right  indicates  display  C  is  operated  by 
control  actions  A  or  B.  Similarly,  two  or  more  displays  (A  and  B) 
operating  concurrently  may  require  the  man  to  perform  a  certain  function. 


or  either  of  several  displays  (A  or  B)  may  require  a  certain  action  of  him. 


However,  only  divergent  branchings  were  used  in  the  model  of  the 
Intercept  Director's  task.  The  forms  of  branching  used  to  indicate 
actions  initiated  by  the  man  were  also  used  to  show  the  relationships 
between  actions  and  displays.  For  example,  control  action  A  may  lead 


to  several  displays  (B  and  C), 


!!)-► 


B 

— 

C 

or 


control  action  A  leads  to  either  display  B  or  display  C  whimsically  on  the 
left,  contingent  upon  previous  conditions  1  or  2  on  the  right.  A  complete 
logical  description  of  the  stimulus -response  sequence  appears  on  drawing 
C-201~2(  all  drawings  appear  in  Appendix  II). 

vVith  the  developed  symbology  it  is  possible  to  represent  a  variety  of 
complex  command  and  control  type  tasks.  By  teaching  the  logic  symbology 
to  a  user,  teaching  of  the  actual  task  can  be  reduced  to  a  "follow-the- 
diagram"  level. 

In  modeling  the  selected  task  a  special  symbolism  for  description  of  the 
complex  visual  display  was  needed.  A  matrix  of  6  x  6  lights  was  chosen  to 
present  the  stimuli  and  the  end  results  of  each  minor  sequence.  The 
diagram  on  the  left  shows  the  arrangement  of  the  lights  in  the  matrix  of  the 
present  model.  The  diagram  to  the  right  shows  the  arrangement  of  blocks 
of  the  SAGE  SID  display. 


E 


A 

A 

C 


4 

6 

4 


Attention  Displays 


SAGE  SID  DISPLAY 
(This  set  of  symbols  represents  a 
single  aircraft  and  its  mission  as 
it  appears  on  the  radar  screen.  The 
information  conveyed  by  each  position 
is  contained  in  the  following  text. ) 


Abstraction,  or  model,  of  the  SAGE  SID  display. 
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In  the  model  display,  the  lights  in  column  A  and  row  VI  later  will  serve 
as  the  "prompting"  stimuli  for  teaching  purposes.  They  are  not  used  in  the 
present  model  of  the  operational  task.  The  remaining  lights  in  the  matrix 
correspond  to  the  SAGE  positions  of  the  symbols.  Because  the  lights  are  in 
two  states  -  on  or  off  -  the  versatility  of  the  conveyed  information  is  reduced  as 
compared  to  the  real  SAGE  displays. 

The  SAGE  symbology  appearing  in  blocks  AjAgAg  and  A^  is  represented 
in  the  model  by  the  lights  IC,  ID,  IE  and  11^  respectively. 

The  letter  symbols  appearing  in  blocks  A^  and  A^  of  the  SAGE  displays 
are  represented  by  the  lights  IC  and  ID  off.  The  numeral  symbols  appearing 
in  blocks  Ag  and  A^  of  the  SAGE  displays  are  represented  by  the  lights  IE  and 
IF  on.  During  the  early  stages  of  the  tracking  mission,  blocks  A^A^A^  and  A 
of  the  SAGE  symbology  convey  information  about  the  track  identity  of  the 
target.  Similarity  is  maintained  in  the  model  through  the  display  of  lights 
IC,  ID,  IE,  and  IF.  In  the  later  stages  of  the  SAGE  tracking  mission  symbols 
in  block  A^  indicate  the  airbase  as  does  the  model  light  IC.  If  the  airbase  is 
open,  the  light  IC  is  on;  if  the  base  is  closed,  the  light  IC  is  off.  Blocks  A^  and 
Ag  of  the  SAGE  system  correspond  to  lights  IIE  and  IIF  in  the  matrix.  In 
the  early  stages  of  the  mission,  when  lights  IIE  and  IIF  are  turned  on,  they 
represent  the  number  of  th  e  Weapon  Director  and  of  the  Intercept  Director, 
respectively.  When  the  light  IIE  is  off,  it  represents  an  unidentified  aircraft. 
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When  the  light  IIF  is  off,  it  represents  an  aircraft  which  is  flying  low.  In 
later  stages  of  the  mission  light  IIF  being  on  represents  an  aircraft  which  is 
flying  high. 

The  Block  E  of  the  SAGE  displays  is  represented  by  the  light  IIB.  If  the 
light  IIB  is  on,  the  aircraft  which  is  being  tracked  is  "on-command"  tracking. 
Otherwise,  the  aircraft  is  "off-command"  tracking. 

The  blocks  Dj  and  D2  of  the  SAGE  symbology  are  represented  by  lights 
IIC  and  IID.  The  enemy  or  target  aircraft  is  represented  by  having  light  IIC 
off.  When  the  light  IIC  is  on,  it  represents  a  friendly  or  an  interceptor  aircraft. 
The  scramble  condition  of  SAGE  procedure  is  represented  by  turning  off  light  IID. 
When  the  aircraft  is  airborne,  the  light  IID  is  turned  on. 

The  blocks  SAGE  symbology  are  represented  by  the  lights 

BIC,  HID,  IHE,  and  HIF,  respectively.  These  lights  represent  the  track 
number  of  the  interceptor.  The  letters  appearing  in  the  track  number  are 
represented  by  the  lights,  which  are  turned  off;  the  numerals,  by  lights  on. 

The  four  "attention"  symbols  of  the  SAGE  symbology  are  represented  by 
lights  IVC,  IVD,  IVE,  and  IVD.  The  first  attention  display  -  ASGN,  assignment 
of  the  task  -  is  represented  by  illumination  of  all  four  lights. 

The  lights  VB,  VC,  VD,  BE,  and  VF  are  used  to  indicate  the  voice 
exchanges  between  the  Intercept  Director  and  the  pilot.  The  first  complete 
radio  contact  is  indicated  by  having  all  five  lights  turned  on.  The  second  step  in 
the  exchange  of  radio  messages  is  indicated  by  lights  VD,  VE,  and  VF. 


The  logic  flow  charts,  instrumented  with  appropriate  pushbuttons,  and 
the  lights  display  matrix  represent  a  simplified  model  of  the  SAGE  Intercept 
Director's  task.  The  operator,  who  manipulates  the  present  equipment,  is 
presented  with  a  series  of  displays.  Drawing  C-201-1  shows  the  sequence  of 
displays  given  to  the  trainee  by  the  hardware  display.  This  sequence  follows 
the  SAGE  mission  through  18  displays.  The  branching  paths  in  the  sequence 
cover  events  which  actually  appear  during  the  tracking  mission  of  the  SAGE 
task.  In  addition,  these  paths  can  be  selected  on  a  probabilistic  basis  thus 
introducing  a  realistic  element  of  uncertainty  about  the  future  developments 
in  the  tracking  mission. 

Drawing  C-201-2  shows  all  of  the  minor  sequences  for  correct  responses 
to  each  of  the  displays.  To  solve  a  given  display,  the  operator  goes  through 
the  appropriate  minor  sequence  of  responses.  When  he  completes  the  series 
of  correct  responses  to  a  given  displayed  problem,  the  display  is  advanced  by 
one  step  to  the  next  minor  sequence,  according  to  the  major  sequence.  This 
changed  display  then  becomes  a  new  problem,  which  in  turn  has  to  be  solved 
by  the  operator  in  a  similar  fashion.  In  the  present  model,  no  provision  has 
been  made  to  change  the  configuration  of  the  display  when  only  partial  responses 
in  the  minor  sequence  have  been  correctly  used.  (An  error  detection  capa¬ 
bility  is  to  be  incorporated  into  the  model  later). 
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Thus,  drawing  C -20 1-2  is  a  flow  diagram  of  the  SAGE  task.  This  flow 
with  branching  contingent  upon  the  display  or  state  of  the  major  sequence 
diagram  shows  the  logical  symbology  which  was  devised  to  analyze  and  describe 
the  SAGE  task  in  terms  of  both  temporal  and  spatial  sequences.  Drawings 
A-201-1  through  A-201-16  show  examples  of  correct  minor  sequences  which 
are  needed  to  complete  given  problems. 

C.  Reduction  of  the  Model  Task  to  an  Operating  Machine 

In  setting  out  to  construct  a  working  device  which  would  embody  the 
model  task  and  which  could  be  manipulated  to  elicit  the  displays  in  their  proper 
order,  several  decisions  on  its  construction  were  made  before  the  detailed 
design  was  begun. 

It  was  decided  to  construct  the  equipment,  insofar  as  possible,  of  modular 
components.  Modular  construction  would  allow  great  versatility  since  any  control 
or  display  element  could  be  positioned  anywhere  within  a  large  area.  Further, 
this  mode  of  construction  did  not  require  as  precise  a  layout  of  the  machine  face 
as  a  fixed  array  would  have. 

Modular  design  (with  the  exception  of  elements  common  to  many  functions 
such  as  power  supply,  stepping  switch,  and  timer)  was  also  applied  to  the 
portion  of  the  device  hidden  from  the  operator.  In  this  case,  the  ease  of 
maintenance  (by  replacement  of  a  faulty  element)  was  a  major  factor  in  the 
decision.  In  addition  to  the  decision  to  fabricate  this  hardware  with  modular 


design,  a  console  was  chosen  to  present  the  displays  and  controls.  This  console 
simulated  the  general  configuration  anticipated  at  operator  stations  of  future 
Information  Systems. 

Several  attributes  of  the  model  task  had  bearing  upon  the  components 
and  circuits  to  be  used: 

-The  major  sequence  of  eighteen  steps 

-The  minor  sequences,  the  length  of  which  is  from  two  to  eleven  steps 

-The  requirement  for  conditional  displays  depending  upon  a  pseudo¬ 
random  factor  with  known  probability 

-The  ambiguity  of  displays  (in  that  their  significance  depends  upon  the 
point  at  which  they  are  encountered  in  the  major  sequence) 

-The  implicit  understanding  that  the  device  might  later  require  error 
detection  and  identification 

-The  fact  that  the  operator^  controls  are  not  assignable  to  specific  steps 
of  the  minor  sequence.  Of  the  30  controls  in  the  task,  eight  are  used  in 
different  minor  sequence  steps  depending  upon  the  major  sequence  status; 
the  remainder  are  unique  to  particular  minor  sequence  steps. 

It  seemed  advisable  also  to  use  a  severely  limited  number  of  different 

circuits  to  accomplish  the  design;  it  was  hoped  that  this  would  facilitate 

trouble-shooting  and  perhaps  make  easier  the  revision  of  the  task,  if  that 

should  prove  desirable. 
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Generically,  the  circuits  used  in  the  machine  fall  into  three  categories: 

1)  latching  circuits,  used  for  memory  and  for  counting 

2)  selection  circuits,  used  primarily  for  control  of  the  display 

3)  a  probabilistic  circuit. 

The  latching  circuits  used  are  shown  on  drawings  B-201-2  and  B-201-3. 

The  self-latching  relay  can  be  energized  by  pushbutton  P  for  example,  and 
will  remain  energized  by  virtue  of  the  ground  path  through  its  own  contact  and 
that  of  relay  X.  Deenergizing  of  relay  1  is  accomplished  either  by  energizing 
relay  X,  thereby  opening  the  ground  path  or  by  pushbutton  which  places  a 
short  circuit  across  the  coil  of  relay  1.  It  should  be  noted,  with  reference  to  this 
circuit,  that  P-^  remains  affected  so  long  as  the  relay  is  energized;  no  further 
use  of  the  pushbutton  can  be  made  until  relay  1  is  released. 

The  second  form  of  latching  circuit  used  is  the  prime  pair.  In  this  case, 
operating  P^  will  energize  relay  2  (via  the  normally  closed  contact  on  relay  1) 
but  will  not  affect  relay  1  until  the  button  is  released,  at  which  time  relays  1 
and  2  are  placed  in  series.  Deenergizing  of  the  entire  circuit  can  be  accomplished 
by  activating  relay  X,  thus  removing  the  ground.  The  prime  pair  has  the 
distinguishing  feature  of  disconnecting  the  pushbutton  from  the  active  portion 
of  the  circuit;  after  both  relays  have  been  energized,  P^  can  be  used,  via  line 
1-1-0,  for  other  purposes.  This  feature  makes  possible  counters  which  register 
the  number  of  times  P^  is  used.  Such  counters,  one  of  which  is  used  to  keep 
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account  of  the  minor  sequence,  require  a  minimum  of  one  relay  per  count 
plus  three  relays  for  the  routing  of  signals.  The  prime  pair  circuit  also  has 
use  as  a  "pulse -stretcher",  yielding  a  contact  closure  which  starts  with  an 
operator's  action  and  ends  only  after  some  particular  relay  fully  operates. 

This  is  the  use  of  the  prime  pair  shown  at  the  bottom  of  Drawing  B-201-3. 

The  major  sequence  control,  shown  partially  in  drawing  B-201-3  is 
simply  a  25-position  stepping  switch  with  its  associated  circuits.  Since  the 
"last"  pushbutton  to  be  used  in  a  minor  sequence  can  be  AB,  XB,  XE,  XG  or 
LG,  it  is  necessary  that  the  stepping  switch  be  allowed  to  choose  the  appro¬ 
priate  one,  and  the  major  sequence  control  circuit  is  arranged  to  do  just  that. 
The  parts  of  this  circuit  which  are  not  shown  in  the  drawing  have  to  do  only 
with  the  means  of  recycling  the  machine  to  an  arbitrary  point  in  the  major 
cycle.  Relays,  R,  S,  T,  U  and  V  represent  one  way  in  which  change  in  the 
major  sequence  state  can  be  inhibited  to  take  account  of  operator  error; 
another,  but  less  selective  way  is  to  have  a  normally  closed  contact  in  series 
with  the  normally  open  contact  of  pole  1-2. 

The  selection  circuit  shown  in  drawing  B-201-1  corresponds  to  the  truth 
table  at  the  top  of  the  drawing.  (Lamp  10  is  really  two  lamps  in  series;  while 
lamp  11  is  a  single  bulb).  Relays  designated  "5,  7,  15,  16,  17"  are  numbered  in 
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accordance  with  five  distinct  displays  in  which  either  lamp  10  or  lamp  11  or 
both  are  used.  As  can  be  seen  from  the  drawing,  both  lamps  have  zero 
potential  across  them  when  none  of  the  five  relays  is  operated.  When  any  one 
relay  is  energized,  one  or  both  lamps  has  the  full  potential  applied,  corres¬ 
ponding  to  the  requirements  shown  in  the  truth  table.  The  relays  which  consti¬ 
tute  the  display  control  are  in  turn  activated  by  being  connected  to  a  deck  of  the 
major  sequence  control  stepping  switch. 

No  drawing  is  given  in  this  report  for  the  circuit  used  to  yield 
probabilistic  effects.  The  circuit  consists  only  of  a  10  rpm  synchronous 
motor,  two  adjustable  cams  and  two  single -pole -double -throw  cam -operated 
switches.  Well  in  advance  of  the  need  for  a  random  choice,  the  motor  is 
started.  The  varying  time  interval  between  the  starting  of  the  motor  and 
the  adjustment  of  the  cams  combine  to  render  the  outcome  unpredictable. 

Yet  the  ratio  of  "switch  closed"  to  "switch -open"  occurrences  is  set  only 
by  the  cam  adjustment  and  is  therefore  known.  If  the  occasion  should  arise 
that  the  time  interval  between  motor  starting  and  switch  output  usage  becomes 
short  or  regular,  then  inclusion  of  two  thermal  time -delay  relays  in  the 
motor  drive  circuit  will  restore  the  pseudo -random  nature  of  the  circuit. 
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D.  Design  of  "Operational"  Console  to  Support  Training  and  Performance 


1.  The  problem  and  its  conceptual  framework. 

The  foregoing  sections  describe  developments  of  the  diagram  form 
and  the  hardware  form  of  the  laboratory  model  which  simulates  the  logical 


structure  of  a  typical  complex  man-computer  task  in  a  Command  and  Control 
System.  This  highly  simplified  model  exemplifies  the  essential  logical 
decisions.  Stimuli  are  reduced  to  light  patterns  and  responses  to  push 
buttons.  Like  typical  Information  System  consoles,  this  one  also  lacks 
performance  aids  and  cues  to  the  logical  structure  of  the  operational  task. 

The  logic  required  to  express  the  complete  task  is  made  up  entirely 


of  the  following  conditions: 


The  "and"  conditions 

(1)  Unordered  "and"  conditions  of  responses  -  the 
man  manipulates  components  of  the  machine  in 
any  order. 

(2)  Sequential  "and"  conditions  of  responses  -  the 
man  manipulates  components  of  the  machine  in 
a  given  sequence. 


The  "or"  conditions 

(1)  Single  choice  "or"  conditions  of  responses  -  the 
man  selects  a  response  which  is  required  by  a 
previous  condition. 

(2)  The  "whim"  condition  of  responses  -  the  man 
selects  one  of  several  equally  satisfactory 
strategies  of  responding. 
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Various  combinations  of  the  above  conditions,  including  the  use  of 
related  symbols  to  show  similar  contingencies  between  actions  and  displays 
were  used  to  construct  a  logic  diagram.  Symbolized  in  this  form,  the  task 
appears  to  be  extremely  simple.  However,  earlier  studies  (26)  on  the  SAGE 
operational  task  upon  which  this  model  was  based,  indicated  that  this  task  was 
difficult  to  learn  and  perform  (1)  when  taught  by  conventional  on-the-job  train¬ 
ing  methods  and  expressed  in  natural  language,  and  (2)  when  no  performance 
aids  or  cues  to  the  operational  task  were  available  at  the  interface. 

In  another  early  study  of  this  same  SAGE  task  (24),  trainees  responded 
well  to  the  new  verbal  programed  instruction  techniques  for  learning  display 
symbology,  concepts,  terms,  etc.  But  they  had  difficulty  understanding  those 
frames  which  taught  the  over-all  task  logic.  It  was  hypothesized  that  this 
difficulty  might  be  related  to  some  undefined  inadequacy  of  the  natural  language 
to  express  unambiguously  and  in  a  compact  form  the  logical  structure  of  these 
man -computer  tasks. 

In  still  another  early  study,  on  a  similar  SAGE  task  (10)  it  was  shown 
that  trainees  learned  the  task  logic  more  easily  and  efficiently  when  they  studied 
from  conventional  text -book  verbal  descriptions.  These  "operator-programs" 
are  compact  diagrammatic  representations  of  the  man's  required  "program" 
within  the  man-machine  task.  They  are  similar  to  the  logic  diagrams  in  this 
study  in  that  relationships  are  represented  graphically  rather  than  verbally. 
However,  these  operator  programs  use  blocks  of  verbal  description  in  place  of 
the  graphical  symbols  which  form  the  nodes  in  the  logic  diagrams  of  this  study 
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Out  of  these  early  studies  grew  the  conviction  that  (1} 
natural  language  and  its  sentence -paragraph  format  is  not  an  efficient 
medium  for  presenting  the  logic  of  complex  physical  systems,  (2)  a 
graphical  language  and  diagrammatic  format  might  be  more  effective. 

Although  there  is  not  yet  any  clear-cut  experimental 
evidence  in  the  psychology  of  learning  to  support  this  conviction, 
engineering  tradition  clearly  supports  it.  To  describe  physical  systems, 
engineers  and  physical  scientists  have  developed  many  graphical  meta¬ 
languages  such  as  mechanical  drawing,  electronic  circuitry  drawing, 
computer -program  flow  diagrams,  energy-bond  diagrams,  etc.  The 
utility  of  these  graphical  languages  and  diagrammatic  forms  in  the 
training  and  work  of  engineers  and  physical  scientists  is  time-tested. 
However,  graphical  languages  have  not  been  widely  used  in  the  training 
and  work  of  the  usual  non-engineer,  non-physical  scientist  operators  of 
complex  physical  systems.  Nor  has  the  possible  utility  of  this  approach 
been  studied  heretofore  even  superficially. 

A  grim  commentary  is  frequently  made  on  the  difficulty  of 
learning  to  operate  today's  complex  Information  Systems:  -  "the  only  ones 
who  can  operate  the  system  effectively  are  the  engineers  who  designed  it.  " 
Undoubtedly,  many  skills  in  the  engineer's  repertoire,  as  well  as  his  depth 
of  knowledge  about  the  system,  contribute  to  the  above  tribute.  Yet  it 
seems  useful  for  the  present  study  at  least  to  entertain  the  assumption 
that  part  of  his  special  skill  stems  from  his  ability  to  study  and  conceptualize 
the  system  dynamics  in  terms  of  graphical  meta-languages  and  concise 
diagrammatic  formats.  It  should  be  relatively  simple  to  equip  the  operator 
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who  is  not  an  engineer  with  this  special  skill.  Development  of  methods 
for  doing  just  this  constitute  the  first  studies  in  this  series. 

The  studies  summarized  below  describe  initial  efforts  in  the 
development  of  a  technique  for  using  a  graphical  language  and  flow 
diagraming  to  mediate  learning  and  support  performance  in  the  operation 
of  the  laboratory  model.  The  task  logic  symbology  and  diagraming  method 
developed  in  the  first  phase  was  used  in  these  studies. 

2.  Console  Design  Features. 

The  next  step  was  to  embody  the  notions  described  above  in  the 
console  design. 

The  response  (or  "working")  panel  of  the  console  mock-up  was  sub¬ 
jected  to  a  major  human  engineering  design  effort.  This  panel  layout  is 
entirely  different  from  that  found  in  typical  operator  consoles  in  current 
Information  Systems.  Figure  1  and  2  show  several  versions  of  this 
approach.  The  essential  new  features  are  as  follows: 

(a)  The  task-logic  diagram  is  an  integral  part  of  the  response 
panel.  Color-coding  is  used  throughout  this  diagram. 

(b)  The  response  switches  are  positioned  within  the  logic 
diagram  where  required.  This  is  in  direct  contrast  to 
the  conventional,  symmetrical  arrangement  of  switches 
wherein  little  regard  is  given  to  their  working  relations. 


(c)  Part  of  the  response  panel  is  given  over  to  an  additional 
"training"  response  panel.  Its  function  is  explained  in 
the  next  section. 

(d Y  "Reinforcement"  lights  are  added.  Their  training  function 
is  explained  in  the  next  section. 

This  design  results  in  a  console  configuration  which  differs  from 
the  conventional  in  that  it  incorporates  training  features  plus  performance 
supports. 

Introduction  of  the  performance  supports  greatly  reduces  the 
complexity  of  the  training  by  eliminating  much  of  the  information  which 
the  trainee  formerly  had  to  learn  and  remember.  On  the  other  hand, 
use  of  the  symbology  and  diagraming  technique  adds  a  new  training 
requirement:  learning  to  interpret  the  graphical  system.  However,  the 
latter  is  a  minor  training  task.  Furthermore,  once  the  trainee  has 
learned  the  system,  he  could  perform  any  task  similarly  diagramed. 

E.  Pilot  Studies 

After  minimum  verbal  instruction  and  several  minutes  of  practice 
naive  subjects  could  trace  through  a  major  sequence  on  the  graphical  flow 
diagram  (Appendix  II,  C-201-2)  without  error,  provided  they  received  imme¬ 
diate  reinforcement  after  each  operation  during  the  practice  period.  The 
minimum  verbal  instructions  consisted  of  the  following  elements: 

1)  task  description;  the  diagram  illustrates  logical  sequence  of  actions, 
you  must  trace  through  the  proper  series. 

2)  starting  point  orientation;  follow  the  diagram  from  left  to  right, 
beginning  with  the  series  labeled  I. 
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3)  reinforcement  orientation;  you  will  be  told  when  any  action 

you  take  is  incorrect  (improper  sequence  or  improper  series). 

Experiments  with  the  hardware  model  were  designed  to  study  training 
procedures  using  the  same  symbolic  and  graphical  task  description 
techniques.  Color -coding  was  used  to  eliminate  the  semantic  content  and 
the  numbering  of  displays  and  contingencies.  In  addition,  preliminary 
observations  of  naive  subjects'  performance  with  the  flow  diagram 
suggested  the  following  specifications  for  the  console: 

1)  ordered  sequence  "and"  buttons  were  placed  directly  along 
the  color-coded  stripes,  rather  than  on  side  branches. 

2)  unordered  "and"  buttons  were  enclosed  in  color-coded  boxes 
rather  than  at  the  end  of  diverging  branches. 

3)  whimsical  branches  were  color-coded  black  to  represent  all 
connecting  paths. 

4)  a  red  light  was  fastened  above  the  working  panel  (see  Figure  1) 
to  provide  negative  reinforcement  after  incorrect  actions. 

The  first  display  presented  to  the  subject  on  the  display  panel  was 
also  symbolically  displayed,  together  with  its  color  code,  on  the  working 
panel  (see  Figure  1).  The  subject's  task  consisted  of  the  following  elements: 
identify  the  color  code  represented  by  the  display,  follow  the  console  diagram 
from  left  to  right  and  activate  all  buttons  in  that  particular  minor  sequence, 
identify  the  color  code  for  the  next  minor  sequence,  etc.  The  complete  major 
sequence  used  in  these  experiments  consisted  of  12  minor  sequences.  As 
each  minor  sequence  was  successfully  completed  the  display  was  advanced  to 
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the  next  minor  sequence. 

Early  emphasis  was  directed  towards  developing  a  self-teaching 
program  requiring  no  written  or  verbal  instruction,  but  such  an  approach 
proved  impractical  to  implement.  The  logic  symbology  used  in  the  working 
panel  of  the  console  was  then  presented  in  a  simplified  form  on  a  "training 
panel",  but  a  small  amount  of  instruction  was  required  even  to  use 
this  instructional  aid.  An  early  form  of  the  console  layout  appears  in 
Figure  1. 

A  small  group  of  subjects  was  run  with  and  without  error 
reinforcement  and/or  the  training  panel.  Observations  from  these 
tests  were  then  used  to  develop  an  improved  console  layout  (see  Figure  2) 
and  accompanying  instructions.  The  early  training  panel  required  following 
color-coded  paths  from  left  to  right,  proceeding  from  the  top  path  to 
the  bottom.  Some  subjects  would  then  attempt  to  work  from  the  top  path 
to  the  bottom  path  when  using  the  working  panel,  although  such  a  progression 
ignored  the  relationships  between  display  and  color-codes.  The  improved 
training  panel  was  organized  differently  and  this  type  of  error  eliminated. 
Although  the  early  working  panel  presented  succinctly  the  logical  aspects 
of  the  task  (a  trivial  task  to  those  who  understood  the  logic  symbology), 
naive  subjects  did  not  approach  it  in  a  logical  manner.  Subjects  tended  to 
search  for  spatial  correspondence  between  lights  and  buttons,  would  repeatedly 
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Figure  1.  An  early  console  layout  showing  working  panel  on  the  left, 
display  in  the  center,  training  panel  on  the  right,  and  the  remote  control 
box  at  the  lower  center.  Red  lights  for  error  reinforcement  are  located 
over  the  two  side  panels. 


Figure  2.  Final  console  layout.  Note  the  simplifications  in  the  first  four 
unordered  "and"  buttons  symbol,  and  in  the  catalog  of  display  color-codes. 
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ignore  the  first  four  unordered  "and"  buttons,  or  would  become  confused 
and  skip  to  the  next  minor  sequence  when  they  encountered  a  whimsical 
branch.  The  improved  console  layout  reduced  these  tendencies,  as 
demonstrated  by  reduced  time  and  errors  during  training  (completion 
of  a  major  sequence  without  errors  was  the  criterion  for  completion  of 
training).  The  first  set  of  instructions  appear  in  Table  1. 

Although  initial  experiments  (three  subjects)  with  the  improved 
console  demonstrated  that  the  task  could  be  learned  with  the  aid  of  the 
training  panel  and  immediate  error  reinforcement,  the  necessity  for 
these  aids  had  not  been  established.  Consequently  a  two  by  two  matrix 
experimental  design  (performance  after  use  of  the  training  panel  versus 
no  such  pretraining,  and  immediate  reinforcement  after  each  action  versus 
reinforcement  only  after  completion  of  minor  sequences)  was  devised 
to  evaluate  these  variables.  Only  two  subjects  were  used  for  each  of  the 
four  conditions.  Subjects  were  Boston  University  students  who  responded 
to  a  request  placed  with  the  Student  Employment  Office. 

Experimental  procedure  was  as  follows: 

1)  the  subject  was  given  a  printed  set  of  instructions  (Table  1) 
and  asked  to  read  them. 

2)  the  experimenter  then  directed  the  subjects'  attention  to  the 
training  panel  or  display  panel  as  required. 

3)  if  the  subject  used  the  training  panel  he  was  required  to  practice 
until  he  could  complete  the  panel  sequence  without  error  (subjects 
generally  reached  this  criterion  on  the  second  try,  and  were  ready 
for  the  main  task  after  a  total  of  about  two  minutes  of  practice). 
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Table  I 


Instructions  used  in  Console  Experiments 


A.  First  set  of  instructions  (immediate  reinforcement) 

1.  Display  Panel.  Many  different  patterns  of  lights  can  appear  on 
the  display  panel.  Your  job  is  to  make  the  first  pattern  of  lights 
change  into  another  pattern.  After  a  certain  number  of  changes  the 
display  panel  lights  will  go  off.  Then  your  job  is  finished. 

2.  Job  Aids.  On  the  left  are  small  colored  pictures  of  the  displays  you 
will  see.  Around  each  picture  is  a  color  strip.  Following  this  color 
will  help  you  to  push  the  correct  series  of  buttons,  which  will  change 
the  display.  If  you  push  a  wrong  button  the  red  light  will  go  on,  to 
help  you  find  the  button. 

B.  Modified  instructions  (immediate  reinforcement) 

« 

1.  Display  Panel,  (same  as  original  instructions  above) 

2.  Job  Aids.  On  your  left  is  another  panel  containing  many  buttons. 

On  the  left  of  this  panel  are  small  pictures  of  the  displays  you 
will  see  on  the  display  panel.  Around  each  picture  is  a  color  strip. 

The  strips  in  black  color  represent  all  colors  on  this  panel.  Following 
a  particular  color  or  a  black  color  from  left  to  right  will  help  you  to 
punch  the  correct  sequence  of  buttons.  Then,  the  display  will  change. 

If  you  punch  a  wrong  button,  the  red  light  above  the  panel  will  go  on. 

Then  go  back  to  the  last  correct  button  and  decide  why  you  made  an 
error. 

The  buttons  on  this  panel  do  not  correspond  in  location  to  the  lights 
on  the  display  panel. 

(if  the  reinforcement  was  delayed  until  the  completion  of  the  minor  sequence, 
the  statement  regarding  the  red  light  was  changed  as  follows:  "If  you  punch  the 
wrong  buttons  anywhere  in  the  left  to  right  sequence,  the  red  light  above  the 
the  working  panel  will  go  on  at  the  Ond  of  the  sequence.  The  display  will  not  change. 
Then,  you  will  have  to  return  to  the  beginning  (left  side  of  the  working  panel)  and 
try  again). 
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4)  if  the  subject  began  immediately  on  the  main  task  he  was  required 
to  continue  on  a  minor  sequence  until  he  completed  it  without  error, 
then  on  to  the  next  minor  sequence  throughout  the  major  sequence. 
The  major  sequences  were  repeated  until  the  subject  performed 
one  without  error  in  any  of  the  12  minor  sequences.  The  subject 
then  performed  a  further  major  sequence  consisting  of  6  minor 
sequences. 

Time  to  completion  and  sequence  of  buttons  activated  were  recorded, 
these  data  being  sufficient  to  define  the  performance.  Results  appear  in 
Table  2. 

The  results  substantiate  the  hypothesis  that  the  logic  symbology  can 
be  taught  on  the  training  panel  and  then  transferred,  to  a  moderate  degree, 
to  the  working  panel.  Use  of  immediate  error  reinforcement  was  found  to 
be  an  effective  substitute  for  pre -training.  That  the  task  requires  some 
form  of  instruction  assistance  was  clearly  demonstrated  by  the  failure  of 
subjects  to  complete  a  single  minor  sequence  without  the  learning  aids. 

The  implications  of  these  experimental  results  for  inclusion  of  self- 
instructional  features  in  Air  Force  Command  and  Control  Systems  equipment 
are  presently  under  study.  These  findings,  together  with  those  from  the 
experiments  described  in  other  sections  of  this  report  will  be  used  to 
develop  general  engineering  principles  useful  in  the  design  of  future  Command 
and  Control  Systems. 
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Table  2 


Time  and  Error  Scores  for  Major  Sequence 
Trials  (12  minor  sequences)  with  and  with¬ 
out  learning  Panel,  and  Immediate  or 
Delayed  Feedback. 

1  Immediate  Reinforcement 


Learning  Panel  No  Learning  Panel 


Subject  Trials 

Time 

Errors 

Subject 

Trial5!' 

Time 

Errors 

A 

1 

7:15 

5 

C 

1  • 

13:30 

15 

2 

5:36 

8 

2 

2:57 

4 

3 

3:33 

0 

3 

2:03 

4 

1:39 

0 

B 

1 

5:45 

25 

2 

3:24 

6 

D 

1 

6:15 

6 

3 

2:30 

0 

2 

2:57 

0 

13  Delayed  Reinforcement 


Learning  Panel 

No 

Learning  Panel 

Subject 

Trial5!' 

Time 

Errors 

Subject 

Trial  Time  Errors 

E 

1 

6:36 

14 

G 

No  major  or  minor 

2 

1:24 

0 

H 

sequences  successfully 
completed  in  the 

F 

1 

18:03 

213 

alloted  time5!'5!'. 

2 

2:51 

0 

^Trials  were  repeated  until  the  subject  performed  a  major  sequence  without 
error. 

**If  the  subject  could,  not  successfully  complete  any  of  the  first  four  minor 
sequences  in  20  minutes,  the  experiment  was  terminated. 
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IV  Studies  on  Information -Solicitation  and  Problem-Solving  Training 

A.  Introduction 

Some  Air  Force  Command  and  Control  tasks  are  at  least  partially 
characterized  by  the  soliciting  of  information  concerning  the  environment, 
and  the  use  of  this  information  to  form  hypotheses  or  to  test  the  validity 
of  previously  formed  hypotheses.  It  is  therefore  of  interest  to  determine 
whether  the  user  of  a  complex  information  system  can  be  taught  by  the 
system  equipment  how  to  solicit  information  efficiently  from  it.  To 
describe  the  capabilities  which  the  system  must  have  in  order  to  provide 
this  tutorial  function,  we  must  know: 

1)  How  to  describe  the  "manner"  in  which  people  query  their 
environment  in  the  process  of  "solving  problems", 

2)  what  portion  of  this  manner  of  solicitation  and  utilization 
of  environmental  information  can  be  deliberately  benefi¬ 
cially  modified, 

3)  how  to  teach  this  (i.  e.  ,  modify  the  natural  behavior), 

4)  how  to  teach  particular  application  of  this  querying- 
reasoning  process  to  particular  people  (in  particular, 
command  and  control  tasks  to  Air  Force  personnel)  ,  and 

5)  how  to  translate  this  tutorial  task  into  system  engineering 
terms  (displays,  controls,  computer  programs,  etc). 

The  obtaining  of  experimental  information  concerning  the  above 
areas  constitutes  a  major  portion  of  the  present  and  planned  programs. 


Initial  experiments  have  been  directed  towards  the  first  three  of  these 
five  areas  using  abstract  "logic  games".  Examination  of  actual  Air 
Force  personnel  performing  Air  Force  command  and  control  jobs  was 
rejected  for  the  initial  effort  due  to  difficulties  in  intelligently  observing 
such  tasks,  generalizing  from  them  to  future  tasks  which  do  not  exist 
today,  and  the  small  number  of  individuals  that  could  be  observed  (and 
therefore  difficulty  in  discriminating  between  individual  "style"  and 
characteristics  invariant  among  successful  performers). 

The  following  were  the  principal  criteria  used  in  selecting  the  games 
to  be  administered: 

1)  Subject's  cognitive  behavior  to  be  manifest  as  much  as  possible. 

2)  His  problem  (i.  e.  his  objective  in  the  game)  to  be  capable  of 
close  specification. 

3)  The  classical  trichotomy  of  logical  processes  -  induction, 
deduction,  and  analogy  would  be  represented  in  the  games 
chosen  (although  analogical  querying-reasoning  would  be 
included  in  inductive  or  deductive  games  rather  than  being 
separately  examined). 

B.  The  Maze  Game 

The  maze  game  is  a  problem  solving  task  in  which  the  subject 
takes  sequential  steps  according  to  certain  rules,  uncovering  clues 
enroute,  and  attempts  to  arrive  at  the  "end"  of  possible  sequences  by 
the  shortest  route.  The  maze  consists  of  a  series  of  39  color-coded 
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tags,  on  the  front  of  which  are  entered  one  or  two  nonsense  syllables. 
Through  manipulation  of  the  clues  necessary  to  lead  to  a  particular  solu¬ 
tion  it  is  possible  to  vary  the  complexity  of  the  task.  Finding  the  logical 
trace  through  the  pattern  of  nonsense  syllables  is  an  inductive  problem. 
The  following  written  instructions  were  given  to  each  of  3  subjects: 
"This  is  like  a  treasure  hunt.  The  clue  that  leads  to  each  tag  is 
given  on  the  front  of  the  tag.  On  the  back  of  each  tag  is  often,  but 
not  always,  a  clue  that  leads  to  some  other  tag.  The  three  red  tags 
are  "free"  clues  that  you  may  use  at  any  time.  Any  of  the  other  36 
tags  can  be  used  only  when  you  already  have  the  clue  or  clues  that 
lead  to  them.  Find  the  tag  which  hides  the  treasure.  " 

There  are  five  equivalent  routes  through  the  maze  which  involve  the 
use  of  only  eight  tags.  One  of  these  routes  is  outlined  below: 


Number 

Front  of  tag 

Clue  on  back  of 

Comments 

tag 

1 

FIFI 

BIP 

"free"  red  lag 

2 

GOKO 

DAK 

"free"  red  tag 

3 

ZADA 

CUG 

"free"  red  tag 

4 

BIP  &  DAK 

QOM, TAH 

5 

QOM  &  TAH 

LYN 

6 

CUG 

RIV 

7 

LYN  &  RIV 

FER 

8 

FER 

JACKPOT 

end  of  maze 
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Use  of  other  tags  lead  to  longer  routes  or  lead  to  blind  alleys.  The 
syllables,  with  several  exceptions,  were  chosen  from  a  list  of  nonsense 
syllables  reported  to  have  low  association  value  (reference  25,  page  540). 

Results  of  6  trials  are  listed  in  Table  3.  Rapid  learning  was  most 
evident  in  the  reduction  of  the  number  of  "blank"  tags  which  were  turned 
over.  Although  the  subjects  demonstrated  the  ability  to  memorize  success¬ 
ful  clues  or  to  discover  the  rules  (or  logic),  it  was  difficult  to  analyze  from 
the  data  how  the  subject  learned  them.  Performance  indicated  that  as  new 
information  was  gained  it  was  "correlated"  with  data  obtained  earlier,  with 
the  next  choice  partially  predicated  upon  all  the  information  on  hand.  Long 
periods  of  study  followed  by  long  periods  of  direct  attempts  at  solution  were 
not  observed  during  these  limited  experiments.  However,  it  is  speculated 
that  different  instructions  or  rewards  for  minimizing  the  number  of  tags 
used  could  alter  these  observations. 

A  modified  form  of  the  game  has  been  programmed  on  the  PDP-1 
computer.1  Through  the  use  of  automated  data  processing  it  may  be  feasible 
to  study  the  effects  of  procedural  variables  on  performance  of  the  maze  game 
by  large  numbers  of  subjects. 
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This  work  is  in  progress  at  Decision  Sciences  Laboratory,  Air  Force 
Electronic  Systems  Division. 
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TABLE  3 


Preliminary  Experiments  Using  the  Maze  Game 


Total  Number  of  Tags 


Subject  I 

A  30 

B  17 

C  21 

Number  of  "Blank"  Tags 
A  8 

B  8 

C  7 


Trials 


II 

in 

IV 

V 

VI 

20 

27 

12 

19 

11 

17 

23 

16 

13 

- 

26 

18 

13 

9 

10 

5 

7 

1 

4 

1 

8 

7 

5 

4 

- 

8 

4 

1 

0 

0 

C.  The  Numbers  Game 

The  numbers  game  was  selected  for  study  since  it  met  all  of  the 
requirements  criteria  and  since  analysis  of  the  subject's  reasoning  can 
also  be  performed.  The  operator's  task  in  the  numbers  game  is  to  deter¬ 
mine  a  four  digit  number  which  has  been  selected  by  the  experimenter. 

The  operator  solicits  information  by  selecting  a  four  digit  test  number  from 
a  list  of  30  such  numbers  and  requesting  a  "score"  for  the  test  number. 

The  score  is  zero  if  the  operator's  number  does  not  contain  any  of  the  exper 
imenter's  digits  in  the  corresponding  positions;  one  for  one  digit  correct, 
etc.  As  the  game  progresses  the  queries  that  the  operator  makes  can  be 
evaluated  as  to  their  pertinence  to  the  decisions  which  he  must  make  and 
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whether  or  not  successive  queries  contain  redundancies.  Tn  addition,  for 
efficient  performance  the  operator  must  sequentially  make  plausible  hypo¬ 
theses  and  then  obtain  confirmation  or  denial  of  them.  Efficient  use  of  both 
positive  and  negative  information  in  the  formulation  and  testing  of  hypo¬ 
theses  leads  to  a  rapid  solution.  The  use  of  sequential  hypotheses  is  in 
contrast  to  the  situation  in  which  an  operator  requests  "all"  the  information 
on  a  broad  subject  at  once.  The  editing  problem  and  likelihood  of  over¬ 
loading  the  operator's  information  handling  capacity  are  high  in  the  latter 
case,  particularly  during  the  learning  phase. 

The  experiments  with  the  numbers  games  were  conducted  in  tv/o 
phases: 

1)  observation  of  15  untrained  subjects  performing  a  long  series 
of  trials  after  reading  a  brief  set  of  instructions.  Eight  of  the 
subjects  received  a  lecture  demonstration  after  five  games  (at 
which  time  the  rules  were  changed  to  permit  the  operators  to 
construct  their  own  four  digit  test  numbers). 

2)  observation  of  three  groups  of  four  subjects  performing  5  games 
after  receiving  selected  forms  of  training  assistance. 

Time,  identity,  and  score  of  each  request  were  recorded.  From  these 

data  it  was  possible  to  reconstruct  each  trial  for  analysis  at  a  later  time. 

Instructional  techniques  used  during  the  second  phase  experiments  were: 

1)  lecture  demonstration  of  the  first  game  using  an  optimal  strategy, 

2)  feedback  of  guiding  comments  after  each  query  during  the  first 
two  games,  ("Socratic"  comments) 

3)  feedback  of  results  during  the  first  two  games  in  terms  of  an 
optimum  logic  bookkeeping  function. 
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The  guiding  comments  used  in  method  2  were  developed  after  an  analysis 
of  common  reasoning  errors  observed  during  performance  by  the  control 
group.  The  ten  comments,  listed  in  Table  4,  were  printed  on  3  x  5  cards 
which  were  then  selectively  displayed  to  the  subject  according  to  the  nature 
of  his  queries.  It  was  anticipated  that  these  comments  would  guide  the 
subject  in  learning  an  optimal  strategy.  The  automated  bookkeeping  function 
was  accomplished  by  updating  after  each  query  a  display  of  all  information 
obtained  from  previous  queries.  Since  all  of  the  logical  deductions  were 
made  for  the  subject,  he  only  had  to  make  shrewd  choices  of  queries  to  reach 
a  solution  with  a  minimum  effort.  It  was  anticipated  that  after  the  first  two 
games  the  subject  would  then  adopt  a  similar  form  of  logical  bookkeeping 
for  the  remaining  games. 

Subjects  were  college  students  who  responded  to  a  request  for  paid 
subjects  to  participate  in  a  psychological  experiment.  The  first  group 
of  15  subjects  were  Boston  University  female  and  male  students.  Other 
groups  were  summer  school  students  (no  attempt  was  made  to  match 
groups  on  the  basis  of  age,  sex,  or  education). 

Results  of  the  experiments  are  summarized  in  Table  5.  After  five 
games  there  were  only  minor  differences  between  groups  in  time  and 
number  of  queries  to  completion.  However,  the  large  number  of  incomplete 
trials  observed  when  using  the  "Socratic" comments  would,  if  included 
greatly  increase  the  magnitude  of  the  time  and  number  of  queries,  observed 
with  this  method.  Using  automated  bookkeeping,  the  subjects'  performance 
on  their  first  trials  were  as  good  as  those  after  five  games  with  any  of 
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Table  4 


Guiding  Comments  Used  as  an  Instructional  Technique 

1.  Good,  one  of  the  four  digits  is  correct.  Ask  questions  which  will 
help  to  find  which  is  correct.  As  you  find  which  are  incorrect  you 
will  also  learn  morel 

2.  Good,  Now  what  numbers  in  which  positions  have  been  ruled  out? 

3.  That  was  a  good  choice.  What  digits  now  remain  as  pc  ssibilities? 

4.  Good,  you  ruled  out  another  possible  digit  and  now  have  more 
refined  information  on  the  previously  scored  numbers. 

5.  Although  this  is  good  information,  don't  forget  to  follow  up  on  the 
scored  numbers  previously  asked. 

6.  Good,  now  you  have  sufficient  information  to  specify  the  digit(s). 

You  can  also  re-evaluate  the  previous  questions  that  had  a  score  due 
to  this  digit.  What  can  you  rule  out? 

7.  Good,  now  you  have  sufficient  information  to  specify  the  complete 
number.  Do  you  see  why?  If  not,  ask  more  test  numbers  from  the 
list,  but  don't  guess  wildly.  Test  hypotheses! 

8.  Poor  choice!  Some  of  the  digits  have  already  been  ruled  out. 

At  this  stage  it  would  have  been  more  efficient  to  select  a  number 
which  will  provide  new  information  on  each  of  the  unknown  digits. 

9.  Bad  choice!  Previously  asked  questions  have  already  given  you  the 
answer  to  that  question.  Do  you  see  why? 

10.  Spatial  correspondence  as  well  as  numerical  correspondence  is 
necessary.  You  have  missed  this  important  point. 
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Table  5 


Summary  Data  from  the  Numbers  Game 
Expe  riments 


Time  to  completion  (minutes  &  seconds) 
for  each  game 


Group 

I 

n 

III 

IV 

V 

Total 

1. 

Control 

mean 

19:11  * ** 

14:53 

19-26 

18-48 

9:57 

16:27 

(n=15) 

median 

18:15* 

13:55 

15:50 

9:30 

9:00 

2. 

First  game 
worked 

mean 

21:17 

15:42 

13:06 

13:11 

15:49 

through 

median 

- 

17:17 

15:30 

12:45 

15:00 

(n=4) 

3.  Comments 


on  games 

1  &2 
(n=4) 

mean 

median 

11:18* 

11:18* 

12:09* 

12:22* 

15:49* 

10:22* 

7:41* 

7:57* 

10:35* 

9:05* 

11:30* 

4.  Automated 
bookkeep- 

mean 

median 

12:10 

12:20 

12:10 

8:47 

21:30 

17:50 

9:55 

8-55 

9:55 

7:57 

13:08 

ing  (n=4) 


Number  of  queries  to  completion 


I 

II 

III 

IV 

V 

Total 

1. 

Control 

mean 

19.0* 

16.9 

18.4 

17.4 

15.6 

17.4* 

median 

18.0* 

15.0 

16.0 

14.0 

15,0 

2. 

First 

mean 

17.2 

17.0 

16.7 

16.2 

16.8* 

worked 

median 

- 

16.5 

15.5 

16.0 

15.5 

3, 

Comments 

mean 

13.0* 

15.7* 

18.5* 

13.5* 

16.5* 

15.4* 

median 

13.0* 

14.0* 

17.5* 

13.5* 

16.5* 

4. 

Bookkeep¬ 

mean 

14.5 

14.2 

21.7 

14,2 

16.0 

16.1 

ing 

median 

13.5 

13.5 

22.0 

15.5 

16.0 

*Four  of  the  15  control  subjects  failed  to  complete  the  first  test  and  are  not 
included  in  the  averages  for  that  test.  Four  cf  six  subjects  receiving  the 
guiding  comments  failed  to  complete  the  first  test,  and  two  of  these  also 
failed  on  the  subsequent  tests.  Therefore  the  reported  figures  represent 
a  biased  population  (the  "good"  subjects)  and  understate  expected  figures. 

**Time  and  questions  for  the  first  test  were  determined  by  the  demonstration 
program  and  are  not  reported  in  the  averages. 
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the  teaching  methods.  Those  subjects  who  received  the  automated  book¬ 
keeping  did  not  immediately  learn  to  perform  this  function  for  themselves, 
as  evidenced  by  the  sharp  increase  in  time  and  number  of  queries  on  the 
third  game.  It  is  notewortny  that  those  subjects  who  observed  a  demon¬ 
stration  of  optimum  strategy  on  tne  first  trial  did  not  perform  better  than 
the  control  group  on  subsequent  trials. 

After  four  games  most  subjects  were  able  to  perform  the  game  with 
only  occasional  deviations  from  an  optimal  strategy.  However,  deductive 
errors  which  led  to  poor  overall  performance  were  occasionally  observed 
after  many  trials  for  subjects  who  had  previously  demonstrated  good 
performance.  Since  instructions  to  the  subjects  were  permissive  (neither 
speed  nor  minimum  numbers  of  queries  were  mentioned),  the  experimental 
results  may  not  apply  in  situations  for  wnich  the  task  statement  differs. 
After  about  five  queries  the  selection  of  an  optimum  test  number  required 
a  careful  search  of  the  list  of  test  numbers.  Consequently,  the  subjects 
had  to  trade  off  time  against  query  effectiveness. 

First  and  fifth  trials  by  a  control  and  by  a  bookkeeping  subject  are 
outlined  in  Tables  6  and  7.  These  subjects  were  not  representative  of 
their  respective  groups,  the  control  being  much  better  than  average  for 
that  group,  and  the  bookkeeping  subject  being  slightly  better  than  average 
for  his  group.  However,  performance  by  these  subjects  illustrate  the 
major  types  Of  errors  observed  with  other  subjects. 

Referring  to  Table  6,  first  game  by  the  control  subject,  the  first  three 
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queries  suggest  that  this  subject  had  already  grasped  the  fundamentals 
of  the  game.  The  subject  selected  trial  numbers  to  attempt  to  pin  down 
which  digit  was  correct  in  the  first  number.  However,  on  the  fourth 
query  the  subject  could  have  been  more  efficient  had  he  selected  a  number 
which  was  either  four  new  digits,  or  which  would  test  which  of  the  last 
three  digits  in  the  second  query  was  correct.  In  the  fifth  query  the  sub¬ 
ject  is  back  on  the  track  following-up  the  second  query.  The  relatively 
long  time  intervals  after  the  second  and  eighth  queries  appear  to  have 
been  used  in  evaluating  past  information  and  determining  which  test  num¬ 
bers)  would  complete  the  information  required  for  a  specific  digit. 

After  the  ninth  query  the  subject  moved  hastily  and  failed  to  fully  evaluate 
the  information  obt  ained  from  the  next  test  number.  Consequently  a  com¬ 
pletely  useless  query  was  made.  After  13  queries  sufficient  information 
had  been  obtained  to  specify  the  complete  four  digit  number,  but  tne 
subject  used  two  more  test  numbers  to  confirm  the  third  digit.  On  the 
fifth  game  the  subject  did  not  make  any  completely  useless  queries,  but  in 
his  haste  he  did  use  four  test  numbers  wnich  were  inefficient.  The  con¬ 
sistent  short  intervals  between  queries  indicates  that  the  subject  had  a 
good  workable  system  of  evaluating  information  content.  The  chance 
selection  of  nine  consecutive  test  numbers  with  zero  scores  is  also 
partly  responsible  for  the  small  time  intervals  since  the  required  logical 
deductions  were  simple. 
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Table  6 


Examples  of  Numbers  Game  Trials  by  Control 
Subject  (AW) 


Test 

Score 

Time 

Comments  * 

Number 

Interval 

1. 

3102 

1 

First  game. 

2. 

3251 

2 

35 

Good,  follow-up  of  first  query. 

3. 

3779 

1 

1 

50 

"  ,  first  digit  established  as  "3". 

4. 

7123 

0 

1 

10 

Fair,  second  digit  has  already  been  ruled  out. 

5. 

1971 

1 

35 

Good,  follow-up  of  second  query. 

6. 

8228 

0 

50 

Fair,  third  digit  has  already  been  ruled  out. 

7. 

4681 

1 

3 

45 

Good,last  digit  established  as  "1". 

8. 

2050 

0 

35 

Fair,  tnird  digit  has  already  been  ruled  out. 

9. 

1564 

1 

1 

05 

Good. 

10. 

4595 

1 

35 

Good,  establishes  second  digit  as  "5". 

11. 

5576 

1 

40 

Poor,  no  new  information. 

12. 

4219 

0 

50 

Good 

13. 

9342 

1 

40 

Good,  complete  number  can  be  deduced. 

14. 

5346 

1 

30 

Poor,  no  new  information. 

15. 

9443 

1 

20 

Poor,  no  new  information. 

16. 

3541 

4 

15 

Success  in  14  minutes  and  15  seconds. 

1. 

8610 

0 

Fifth  game. 

2. 

2788 

0 

25 

Good,  all  new  information. 

3. 

0965 

0 

25 

Good,  all  new  information. 

4. 

3102 

0 

20 

Good,  all  new  information. 

5. 

5967 

0 

25 

Fair,  2nd  &  3rd  digits  already  ruled  out. 

6. 

1034 

0 

40 

Good. 

7. 

4219 

0 

25 

Fair,  3rd  digit  already  ruled  out. 

8. 

6486 

0 

25 

Fair,  3rd  digit  already  ruled  out. 

9. 

7892 

0 

30 

Good,  first  digit  established  as  "9". 

10. 

9342 

1 

25 

Good,  second  digit  established  as  "5". 

H. 

8228 

0 

30 

Fair,  last  digit  already  ruled  out. 

12. 

3251 

2 

25 

Good,  complete  number  can  be  deduced. 

13. 

9551 

4 

30 

Success  in  6  minutes  and  25  seconds. 

sic 


The  comments  represent  an  evaluation  of  performance  and  were  not 
given  to  the  subject. 
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Table  7 


Examples  of  Numbers  Game  Trials  by  Subject  (KU) 

Who  Received  Instruction  Through  Automated  Bookkeeping 


Test  Score  Time  Comments  * 


Number 

Interval 

1. 

5967 

0 

First 

game  (with  bookkeeping). 

2. 

3102 

1 

35 

Good. 

all  new  information. 

3. 

1564 

1 

30 

Fair, 

3rd  digit  already  ruled  out. 

4. 

3251 

2 

50 

Good, 

follow-up  of  second  query. 

5. 

3779 

1 

1 

20 

Good , 

1st  digit  established  as  "3". 

6. 

5576 

1 

1 

05 

Good, 

2nd  digit  established  as  "5". 

7. 

3551 

Wrong 

1 

05 

Poor. 

fourth  query  ruled  out  this  guess. 

8. 

2050 

0 

1 

00 

Good, 

4th  digit  established  as  "l". 

9. 

4219 

0 

1 

00 

Good. 

10. 

8228 

0 

45 

Good . 

11. 

7137 

0 

30 

Good. 

12. 

5346 

1 

25 

Good, 

complete  number  can  be  deduced. 

13. 

3541 

4 

15 

Success  in  nine  minutes  and  twenty  seconds. 

1. 

2788 

0 

Fifth  ; 

game . 

2. 

9007 

1 

20 

Good, 

all  new  information. 

3. 

9443 

1 

35 

Good , 

follow-up  of  second  query. 

4. 

8610 

0 

50 

Good, 

all  new  information. 

5. 

5967 

0 

35 

Good, 

follow-up  of  second  query. 

6. 

3102 

0 

25 

Good, 

follow-up  of  second  query. 

7. 

4219 

0 

25 

Fair, 

3rd  digit  already  ruled  out. 

8. 

6850 

1 

40 

Fai  r . 

4th  digit  already  ruled  out. 

9. 

7892 

0 

25 

Good, 

follow-up  of  previous  query. 

10. 

3251 

2 

30 

Good, 

1st,  3rd,  &  4th  digits  established  (9-51 

11. 

1971 

1 

50 

Poor, 

fifth  query  answered  this. 

12. 

5346 

0 

35 

Good. 

13. 

1564 

1 

25 

Good, 

complete  number  can  be  deduced. 

14. 

9551 

4 

15 

Success  in  six  minutes  and  50  seconds. 

*  The  comments  represent  an  evaluation  of  performance  and 
were  not  given  to  the  subject. 
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Referring  to  Table  7,  first  game  by  the  bookkeeping  subject,  the 
first  three  queries  do  not  suggest  any  keen  insight  into  the  game  strategy. 
However,  the  automated  bookkeeping  display  or  other  undetermined  factor 
guided  the  subject  back  to  following-up  the  second  and  third  queries.  After 
six  queries  the  subject  demonstrated  a  lack  of  understanding  by  guessing--- 
an  impossible  guess  since  query  number  four  would  have  had  to  be  scored 
M3"  for  the  guess  to  be  correct!  However,  with  all  logical  deductions 
made  automatically  it  was  a  simple  task  to  complete  the  trial  efficiently. 

On  the  fifth  game  this  subject  made  one  completely  ineffective  query 
(number  eleven)  and  only  two  that  were  less  than  optimal.  Poor  luck  in 
those  randomly  selected  test  numbers  made  the  task  difficult,  but  the 
subject  demonstrated  a  good  understanding  of  the  optimal  strategy. 

An  evaluation  of  the  trials  by  other  subjects  who  received  the 
automated  bookkeeping  indicated  that  all  subjects  learned  to  use  the 
zero  scores  effectively,  but  several  had  considerable  difficulty  in  making 
logical  deductions  which  would  allow  them  to  evaluate  the  scored  test  num¬ 
bers  effectively. 
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V.  The  Phylogenic  Concept  of  Program  Order 

A.  Background  of  the  Problem 

One  of  the  major  ambiguities  in  the  current  state-of-the-art  of 
programed  instruction  and  automated  training  is  the  question  of  how  to 
organize  and  order  content  in  the  teaching  program.  The  only  available 
answer  now  is  "use  a  logical  order.  "  This  rule  of  thumb  leaves  much 
to  be  defined  -  even  for  the  highly  verbal,  academic  subject-matter 
programs  from  which  it  originated.  However,  as  in  the  present  case 
where  the  programed  instruction  is  concerned  with  teaching  utilization 
of  complex  physical  systems,  the  "logical  order"  rule  is  of  little  help. 

This  chapter  considers  the  special  problem  of  content  order 
where  the  program  is  concerned  with  teaching  utilization  of  a  complex 
physical  system  made  up  of  multiple  subsystems.  It  explores  one 
possible  conceptual  approach  and  examines  the  implications  of  the  proposed 
sequencing  rule.  Implementation  of  this  concept  would  be  reflected  in 
the  "availability"  to  the  trainee  of  different  system  controls  at  various 
stages  of  training.  Problems  would  lead  to  the  discovery  of  system 
functions.  Within  each  level  of  the  program  the  problems  would  be  graded 
with  respect  to  allowable  error  tolerances  in  the  use  of  each  function. 

This  method  would  shape  precise  skills  for  utilizing  each  function. 

The  proposed  phylogenic  concept  leads  to  training  by  a  "method  of 
discovery.  "  It  would  be  implemented  in  the  form  of  a  combination  of 
training  program,  problems  and  equipment  features  which  would  lead  the 
trainee  automatically  along  the  paths  of  such  discovery.  This  teaching 
method  is  similar  to  the  "discovery  method"  being  proposed  as  part  of  a 
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dramatic  reform  in  the  school  mathematics  curriculum  (28).  Like  the 
latter,  this  phylogenic  concept  for  systems  training  aims  to  produce 
analytic  power  and  knowledge  of  capacities  and  limitations,  rather  than 
reliance  on  "formula"  solutions. 

This  phylogenic  concept  seems  particularly  promising  for  the  type 
of  training  required  in  those  future  Information  Systems  which  will  be 
implicitly  programed. 

B.  Introduction 

It  is  desired  that  techniques  be  developed  whereby  existing  and  future 
systems  which  require  operators  may  be  instrumented  or  altered  in  such 
a  way  that  they  may  teach  the  operator  his  role  without  preliminary  or  con- 
concurrent  outside  instruction.  The  systems  to  which  the  techniques  are  to 
be  applied  may  range  in  complexity  from  those  demanding  the  simplest 
mechanical  operations  to  those  that  exist  only  on  an  abstract  or  adminis¬ 
trative  level  such  as  the  decisions  attendant  to  the  choice  of  tactics  in  a 
strategic  situation.  The  approach  discussed  in  this  portion  of  the  report 
relates  an  attempt  to  generalize  upon  the  abstract  problems  to  be  encountered 
in  the  implementation  of  self-teaching  processes.  A  suggested  approach  to 
a  constructible  analogue  or  prototype  is  described  at  the  end. 

It  is  necessary  at  the  outset  that  certain  observations  be  made  regarding 
the  nature  of  the  operator,  his  role  in  the  system,  the  system  itself,  and 
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its  role  in  a  larger  environment  upon  which  it  is  to  make  measurements 
and  produce  alterations.  First:  we  shall  assume  that  since  an  operator 
is  required,  his  role  is  one  which  cannot  be  automated,  at  least  at  present. 
That  is,  we  are  considering  systems  to  which  the  utmost  in  computer 
technology  has  already  been  applied  in  the  processing  of  inputs  and  the 
conditioning  of  outputs.  Somewhere  in  the  loop  there  is  necessitated  a 
filtering  function  which  only  a  human  operator  can  subsume.  Therefore, 
second:  the  information  presented  to  the  operator  has  a  semantic  content 
and  the  controls  available  to  him  must  have  an  effect  which  ultimately 
alters  the  sources  of  that  kind  of  information.  There  may  also  be  avail¬ 
able  to  the  operator  various  means  of  weighting  or  reassembling  the 
incoming  information  as  an  aid  to  decision  or  memory  and  he  must  be 
taught  the  proper  use  of  these  tools  also.  Third:  the  intrinsic  nature  of  the 
learning  process  is  bound  to  be  different  from  the  ultimate  operating  pro¬ 
cess  so  that  it  may  be  necessary  both  to  alter  initially  the  system  functions 
being  taught  and  to  train  the  subject  in  the  learning  process  itself  at  the 
outset. 

C.  The  Operator,  the  System,  and  the  Environment 

A  useful  way  to  visualize  schematically  the  operator,  the  system,  and 
the  environment  is  in  the  form  of  three  respectively  linked  structures  or 
networks  between  which  one  is  trying  to  achieve  an  impedance  match.  The 
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system,  which  holds  the  central  position  between  the  other  two,  has  been 
designed  to  provide  the  proper  match  and  to  allow  each  to  communicate 
with  the  other  as  efficiently  as  possible.  However,  fortunately  or  unfor¬ 
tunately  for  the  problem  at  hand,  the  system  and  the  environment  both 
exist  already,  and  the  naive,  untrained  subject  may  be  considered 
initially  to  present  a  total  mismatch.  It  is  the  purpose  of  the  training 
program  to  adjust  his  output  impedance  to  match  the  input  impedance  of 
that  end  of  the  system  which  is  available  to  him.  The  usual  training  pro¬ 
cess  consists  of  making  many  preliminary  coarse  adjustments  in  the 
operator  before  he  is  placed  "on-line",  allowing  the  fine  match  to  be 
achieved  in  the  latter  condition.  However,  if  a  system  is  to  be  made 
self-teaching,  considerable  alteration  of  the  system  or  of  the  apparent 
environment  or  both  may  be  necessary  in  order  for  learning  to  commence. 
Once  the  process  has  started,  the  structures  may  gradually  revert  to  their 
"normal"  state  as  the  self-adapting  property  of  the  human  operator  allows 
him  to  follow  and  adjust  his  impedance  appropriately.  The  questions  that 
must  be  addressed  here  concern  the  proper  starting  point  and  the  proper 
management  of  the  ensuing  development. 

D.  Function  vs.  Structure 

In  considering  any  given  system  one  might  ask  oneself  who  it  is  that 
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knows  that  system  best  both  functionally  and  mechanically.  The  best 
answer,  it  would  seem,  is  that  it  is  he  who  originally  designed  the  system. 
Why?  Because  each  subcomponent  of  the  system  was  designed  to  fulfill 
a  specific  functional  need  and  for  the  designer  it  was  at  the  height  of  his 
apprehension  of  that  functional  need  that  he  invented  that  subcomponent.  If, 
then,  we  can  elicit  the  same  sort  of  inventive  process  from  the  trainee,  and 
can  present  to  him  a  display  or  control  only  after  he  has  apprehended  its 
necessity,  there  should  result  an  operator  who  has  a  firm  grasp  of  the 
use  of  the  system.  Note  that  we  are  not  asking  him  to  design  the  system 
mechanically,  only  functionally,  for  once  he  has  realized  his  need  for  a 
specific  kind  of  information  or  control,  we  may  then  present  him  with  the 
actual  system  component  which  exists  for  the  purpose  and  with  a  little 
experimenting  on  his  part  he  will  find  that  his  wish  has  been  granted.  The 
means  by  which  the  inventive  process  may  be  stimulated  and  the  manner  in 
which  the  subject  verbalizes  his  requests  are  problems  which  will  be  taken 
up  presently. 

It  should  be  pointed  out  here  that  we  are  not  advocating  a  manhunt  for  the 
actual  designer  himself,  because  in  fact  there  probably  was  no  one  such 
person.  We  are  saying,  rather,  that  it  is  the  creator's  role  and  attitude 
that  we  are  trying  to  elicit  from  the  subject.  We  may  therefore  visualize 
our  own  reconstruction  of  the  evolution  of  the  system  and  it  may  take  any 
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number  of  equally  useful  logical  forms. 

It  is  apparent  that  any  such  teaching  process  in  which  the  operation 
being  taught  is  allowed  to  develop,  piece  by  piece,  at  the  command  of  the 
trainee,  is  inherently  self-adapting  to  the  individual.  The  training  program 
will  eventually  have  led  him  to  the  development  of  the  entire  system  but 
along  the  way  his  requests  for  new  subcomponent  displays  and  controls 
will  have  been  conditioned  to  some  extent  by  his  current  understanding  of 
the  problems  before  him  and  of  the  components  already  at  hand.  However, 
inasmuch  as  the  training  process  differs  from  the  final  operating  process, 
one  must  not  expect  that  the  geometry  of  the  system  environment  will  be  a 
real  one.  That  is,  the  elemental  and  incremental  problems  presented  to 
the  trainee  may  be  considerably  distorted  in  their  time  and  space  dimen¬ 
sions  from  problems  of  a  similar  nature  which  might  occur  during  actual 
operation  of  the  system.  For  example,  an  "emergency"  situation  would 
not  result  in  disastrous  consequences  if  coped  with  slowly:  its  semantic 
content  of  emergency  must  be  taught  before  urgency  may  be  expected. 

E.  Phylo genes  is 

In  coming  now  to  the  consideration  of  how  best  to  proceed  in  the  pre¬ 
sentation  of  the  system  functions  to  the  subject,  we  return  to  the  proposition 
stated  above  that  it  is  the  "original  designer"  of  the  system  who  knows  it 
best.  If  we  are  seeking  a  logical  order  of  development,  what  better  one 
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might  there  be  than  the  process  of  evolution  through  which  the  system  itself 
has  passed  in  arriving  at  its  present  form?  Hopefully  it  is  reasonable  to 
assume  that  each  new  modification  applied  to  the  system  in  the  past  was 
in  fact  an  improvement  which  broadened  the  scope  of  display  or  control 
available  to  the  operator,  and  was  the  result  of  a  desire  to  solve  a  problem 
for  which  the  previously  unmodified  system  was  inadequate.  One  would 
turn,  then,  to  the  "phylogeny"  of  the  system  for  the  teaching  program. 
Addition  or  deletion  of  steps  may  occasionally  be  necessary  because,  on 
the  one  hand,  systems  are  often  updated  in  many  ways  at  once,  and  on  the 
other,  functions  which  were  formerly  included  may  have  atrophied  and 
been  discontinued  later.  However,  in  most  cases  the  phylogenic  history 
of  the  system  will  be  an  excellent  basis  for  the  program  and  it  is  available 
in  the  technical  archives  of  the  developer.  We  repeat,  however,  that  the 
actual  phylogeny  is  not  really  necessary.  It  is  only  proposed  that  it  may 
have  been  a  most  reasonable  sequence  of  steps  on  which  to  base  the  train¬ 
ing  program. 

F.  Example  No.  1 

It  would  perhaps  be  most  instructive  at  this  point  to  indicate  by  example 
the  application  of  the  phylogenic  teaching  method  to  two  systems  that  are 
relatively  familiar:  first,  the  driving  of  an  automobile.  It  will  be  assumed 
for  the  purposes  of  exposition  that  the  problem  of  man-machine  communica¬ 
tion  has  been  solved  sufficiently  for  the  subject  to  verbalize  his  requests 
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for  functions  in  an  appropriate  machine  language.  The  assumption  is  not 
a  trivial  one  but  for  the  moment  we  m’ight  allow  the  existence  of  an  instruc¬ 
tor  as  an  interpreter  with  the  understanding  that  he  is  to  be  eliminated 
eventually. 

Training  commences  with  the  subject  being  placed  in  an  automobile 
devoid  of  any  controls  or  dashboard  displays  and  situated  at  the  top  of  a 
long  hill.  He  is  told  only  that  his  goal  is  to  transport  himself  to  a  visable 
point  "over  there",  and  the  automobile  is  given  a  nudge  to  start  it  rolling 
down  hill.  Depending  upon  the  geometry  of  the  road,  the  subject's  first 
request  might  be  for  a  steering  function  or  it  might  be  for  something 
which  allows  him  to  slow  or  stop  himself  at  will.  Let  us  assume  the 
latter,  but  notice  the  illustration  that  the  order  of  requests  may  be  influ¬ 
enced  by  our  choice  of  environmental  geometry.  The  subject  will  now 
ask  for  a  control  which  will  allow  him  to  slow  the  automobile  whenever 
he  wishes  but  he  need  only  verbalize  his  demand  for  a  slowing  functionj 
he  is  not  required  to  invent  the  structure  of  a  foot  pedal  or  a  hand  brake. 

In  response  to  his  request  a  foot  pedal  appears  and  since  it  is  obviously 
to  be  used  by  the  foot,  he  tries  it  and  soon  is  satisfied  that  his  request 
has  been  granted.  Next  a  curve  appears  in  the  road  and  the  subject 
realizes  that  a  turning  function  is  necessary.  Again,  he  does  not  need 
to  specify  form:  steering  wheel,  joystick,  or  rudder  pedals;  he  has  only 
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to  verbalize  the  function.  A  steering  wheel  is  revealed  to  him  and  with  a 
little  experimenting  he  finds  that  it  will  produce  the  desired  effect. 

Presently  the  automobile  arrives  at  the  bottom  of  the  hill  and  the 
means  of  self-propulsion  must  be  sought.  The  order  in  which  the  discovery 
or  invention  of  accelerator,  clutch,  gear-shift,  ignition  key,  et  cetera  are 
elicited  by  the  teaching  program  depends  upon  the  proper  manipulation  of 
the  environment  and  the  latter  contains  problems  which  had  best  remain 
hidden  at  this  stage.  However,  the  example  has  been  carried  far  enough 
to  demonstrate  a  rudimentary  application  of  the  method.  Note  that  the 
possibility  of  considerable  time  and  space  distortion  has  already  arisen 
depending  upon  the  individual  subject.  If  he  is  an  adventurous  type,  he  may 
not  have  requested  a  slowing  function  until  he  has  reached  a  high  speed  or 
even  until  a  stone  wall  has  been  placed  in  front  of  him  to  create  an  emer¬ 
gency.  Thereupon  the  stone  wall  will  have  to  be  held  at  bay  long  enough 
for  him  to  verbalize  his  request  and  experiment  with  the  foot  pedal  pre¬ 
sented.  In  the  extreme  case  one  might  have  to  let  him  crash  into  the 
wall  and  then  start  him  over  again  at  the  top  of  the  hill  in  order  to  convey 
the  meaning  of  speed  to  him.  The  term  "semantic  content",  as  used  in 
this  discussion,  implies  that  a  control,  a  display,  or  an  environmental 
situation  must  have  meaning  to  the  subject,  and  the  lengths  to  which  the 
program  must  go  to  impress  that  meaning  upon  the  subject  may  vary 
greatly  with  the  individual. 
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G.  Example  No.  2 


As  a  second  illustration  of  the  phylogenic  method  let  us  consider  the 
teaching  of  standard  business  practice.  Certainly  any  naive  trainee  starts 
with  some  crude  notion  of  the  operation  of  a  company;  he  will  be  asked  to 
describe  or  diagram  that  notion  and  will  be  told  that  his  goal  is  the  develop¬ 
ment  of  the  detailed  structure  of  a  large  corporation.  Starting  with  the 
individual's  simple  diagram,  problems  in  bookkeeping,  inventory  manage¬ 
ment,  capital  flow,  salary  schedules,  and  so  forth  will  be  thrown  at  him 
in  such  a  way  that  each  new  problem  requires  a  logical  addition  to  his 
diagram.  In  order  for  him  to  grasp  fully  the  meaning  and  uses  of  each 
addition,  its  invention  must  be  elicited  from  him  with  little  or  no  prompting. 
This,  then,  is  postulated  as  one  of  the  major  advantages  of  the  phylogenic 
method:  that  necessity  is  the  mother  of  invention  and  that  the  creative  pro¬ 
cess  of  invention  produces  the  strongest  and  most  meaningful  impression  of 
the  purpose  of  a  system  component. 

In  neither  of  the  two  examples  was  it  necessary  actually  to  trace  the 
evolution  of  the  giv  en  system.  For  the  purposes  of  driver  training  the 
details  of  engine,  tire,  chassis,  or  body  design  were  irrelevant;  and  in  the 
teaching  of  business  practice  we  did  not  research  the  history  of  mercantil¬ 
ism.  In  both  cases  it  was  necessary,  rather,  to  put  ourselves  in  the 
place  of  the  subject  and  to  imagine  how  he  would  view  his  environment 
and  how  he  should  verbalize  solutions  to  the  "modern"  problems  we  will 
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present  to  him. 

H.  Discussion 

Let  us  now  examine  some  of  the  properties  of  the  special  problems 
involved  in  the  application  of  the  phylogenic  teaching  method.  A  means  of 
communications  must  be  set  up  between  the  subject  and  the  system  whereby 
he  may  make  requests  for  functions  not  yet  available;  the  semantic  content  of 
environmental  situations  must  be  conveyed  and  be  made  interpetable  by  the 
system  displays;  and  the  invention  of  the  function  of  every  control  on  the 
final  system  must  be  elicited  before  training  is  complete. 

In  the  cases  where  a  subcomponent  has  multiple  functions,  they  must  be 
separated  during  training  to  allow  them  to  be  discovered  at  the  appropriate 
time.  More  important,  however,  is  the  requirement  that  the  trainee  be 
taught  at  the  outset  how  to  abstract  the  simplest  possible  verbalization  of 
the  function  he  desires.  If  he  himself  is  unsure  of  exactly  what  it  is  he 
wants,  his  understanding  of  what  has  been  given  him  will  be  incomplete.  To 
some  extent  the  form  of  the  request  made  will  depend  upon  the  manner  of 
problem  persentation,  but  the  simplification  of  requests  may  well  have  to 
be  taught  by  preliminary  instruction  in  the  application  of  the  method.  Con¬ 
sideration  will  be  given  to  the  necessity  for  a  generalized,  aboriginal  teach¬ 
ing  device  for  this  purpose  later. 

The  development  of  the  first  example,  driving  an  automobile,  was  ter¬ 
minated  before  it  had  dealt  with  the  uses  of  the  gear  shift  or  the  ignition 
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key,  which  operations  differ  from  those  of  brake  or  steering  wheel  in  that 
the  former  involve  discrete  functions  while  the  latter  produce  a  continuous 
effect  upon  the  system  in  its  environment.  The  postulation  that  a  subject  is 
to  determine  by  experiment  whether  a  requested  control  has  been  granted  to 
him  makes  it  clear  why  such  "continuous -effect"  devices  are  easier  to  deal 
with  in  a  teaching  program.  They  allow  the  subject  to  apply  perturbations 
with  a  magnitude  and  direction  of  his  own  choosing.  That  is,  his  use  of  them 
is  neither  right  nor  wrong  but  their  functions  become  obvious  immediately. 

In  teaching  the  use  of  controls  whose  operations  are  of  a  discrete  nature, 
one  must,  as  before,  arrange  to  elicit  from  the  subject  the  desire  for  such 
controlling  functions  but  it  should  be  made  clear  to  him  either  before  he 
makes  his  request  or  after  the  control  has  been  presented  that  its  operation 
is  binary  or  multivalued  rather  than  continuous.  The  kind  of  perturbations 
he  can  produce  with  this  control  will  perhaps  be  less  descriptive  of  its  pur¬ 
pose,  so  one  must  be  careful  to  provide  only  the  function  specifically  requested. 

There  may  often  arise  situations  in  which  the  subject  requests  functions 
which  are  either  unavailable  or  irrelevant.  For  instance,  having  been  told  when 
he  is  placed  in  the  automobile  that  his  goal  is  to  arrive  eventually  "over  there", 
he  might  ask  for  a  means  by  which  to  raise  himself  off  the  ground  and  fly. 

This  request  would  not  be  irrelevant  if  he  were  being  taught  to  fly  an  airplane 
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but  it  is  certainly  not  allowed  for  automobiles.  In  such  cases  it  would  seem 
perfectly  logical  either  to  ignore  or  refuse  the  request  provided  that  it  is 
not  one  which  would  be  honored  at  a  later  time.  Some  indication  of  irrelevancy 
or  forehandedness  may  have  to  be  included  in  a  self-teaching  device. 

Before  concluding  the  discussion  of  the  examples,  some  attempt  should 
be  made  to  describe  a  useful  automation  of  the  man-machine  communication 
problem.  In  the  case  of  the  automobile  one  might  provide  the  subject  with  a 
miniature  model  of  his  vehicle  and  the  environment.  By  grasping  the  model 
automobile  he  would  cause  it  to  execute  the  kind  of  maneuvers  for  which  he 
desires  an  effective  control  on  his  own  car  and  would  thereby  communicate 
the  function  wanted  to  the  system.  He  would,  of  course,  be  constrained  from 
performing  multiple  maneuvers  simultaneously  or  from  attempting  impossible 
ones. 

1  .  The  Aboriginal  Machine  or  Primer 

It  was  pointed  out  in  the  initial  arguments  above  that  the  naive  trainee 
may  be  considered  at  first  to  present  a  total  impedance  mismatch  to  the 
system  whose  operation  he  is  to  learn.  If  he  is  to  take  even  .the  first  step 
toward  "inventing"  the  system  for  himself  he  must  already  have  a  grasp  of 
the  rudiments  of  the  man-machine  language  with  which  he  is  to  verbalize 
his  requests.  It  would  seem  necessary,  then,  that  the  trainee  be  given 
preliminary  instruction  in  the  language  and  in  the  general  rules  of  application 
of  the  phylogenic  method  of  system  development.  But  inasmuch  as  it  is  a 
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learning  process  that  must  be  taught  —  a  process  which  does  not  differ 


much  between  systems  to  which  it  will  be  applied  —  the  development  of 
an  aboriginal  machine  or  primer  is  proposed  to  accomplish  this  end. 

All  trainees,  for  whatever  operating  roles  they  may  be  destined,  would 
commence  their  training  on  the  primer;  and  it  would  continue  to  be 
available  to  them  later  for  occasional  review  of  the  method  as  needed. 

The  actual  form  of  the  primer  has  not  yet  been  conceived  but  it  is 
visualized  tentatively  as  an  electronic  and  mechanical  device  offering 
knob  and  switch  controls  and  meter  and  scope  displays.  The  "environment" 
would  be  a  two-or  three-dimensional  model  to  which  the  subject  would  have 
direct  access,  both  for  observation  and  for  manual  perturbations. 

J.  Updating  of  Systems 

A  final  note  on  the  advantages  of  the  phylogenic  method  regards  the 
application  of  the  technique  to  the  retraining  of  operators  on  updated  systems. 
Since  one  may  logically  assume  that  any  updating  of  a  system  involves 
improvements  of  necessary  additions  to  displays  or  controls,  all  that  is 
required  of  the  phylo genetically  trained  operator  is  that  he  carry  his 
training  one  or  more  steps  further  in  a  process  which  he  has  already  learned. 
If  there  has  been  a  considerable  time  lapse  since  his  original  training,  a 
review  of  the  method  on  the  primer  and  perhaps  a  quick  re-invention  of  the 
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system  he  has  been  operating  will  suffice  to  prepare  him  for  learning 
the  uses  of  the  new  improvements. 
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K.  Prototype  Model  for  Demonstration  of  Phylogenic  Teaching 

A  hardware  model  has  been  constructed  for  demonstration  and 
research  in  phylogenic  teaching  (see  Figure  3).  Using  this  device  the 
subject  is  taught  to  control  the  characteristics  of  a  Lissajou  figure  on 
an  oscilloscope.  The  console  contains  all  of  the  controls  and  figure 
generation  circuits,  including  power  supply. 

The  console,  as  it  is  first  presented,  is  a  flat  surface  with  holes  in 
it  showing  the  ends  of  shafts  or  toggles  just  out  of  reach.  Each  hole  is 
color  coded.  The  desired  Lissajou  figure  is  displayed  on  a  transparent 
overlay  in  front  of  the  oscilloscope.  The  subject's  task  is  to  manipulate 
the  scope  figure  so  that  it  coincides  with  the  figure  on  the  overlay.  Start¬ 
ing  with  an  ellipse  for  both  figures,  but  with  the  scope  figure  off  to  the 
right,  the  subject  requests  a  control  which  will  allow  him  to  shift  the 
figure  to  the  left.  Similarly,  other  figures  are  presented  which  introduce 
the  subject  to  vertical,  gain,  "limiting"  (flattened  area),  "roundness", 
and  axis  reversal  controls  as  they  are  requested.  As  the  figures  increase 
in  difficulty  a  complete  understanding  of  the  functions  of  previously  used 
controls  is  necessary  for  rapid  progress.  Provision  has  been  made  for 
display  of  two  separate  figures  on  the  oscilloscope  through  use  of  a  time 
sharing  circuit  and  a  second  set  of  controls.  All  controls  act  independently 
of  each  other. 

Additional  details  on  this  method  are  provided  in  Appendix  III  and  IV. 
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Figure  3.  Console  and  oscilloscope  display  for 
demonstration  of  the  phylo genic  teaching  concept. 
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VI.  Conclusions 


The  first  year's  effort  on  this  project  has  produced  the  following 
conclusions  in  the  three  study  areas: 

A.  Method  for  Self-Instruction  in  Task  Logic 

1.  The  essential  features  of  a  representative  command  and 
control  task  can  be  modeled  with  the  new  analytic  symbology  devised 
for  this  study. 

2.  This  graphical  symbology  provides  a  concise  language  which 
is  an  efficient  medium  for  teaching  the  task  logic. 

3.  The  logic  flow  diagrams  constructed  with  this  symbology 
can  be  used  as  self-teaching  programs.  Exploratory  studies  indicate  use 
of  this  symbolic  language  reduces  some  of  the  disadvantages  of  a  natural 
language  (English)  found  in  earlier  programs  designed  to  teach  task  logic. 

4.  Design,  layout  and  labeling  of  console  response  panels  in  a 
topological  relation  to  the  task  logic  flow  diagram  appear  to  hold  promise 
of  facilitating  the  self-teaching  function. 

B.  Information-Solicitation  and  Problem-Solving  Training 

1.  An  experimental  task  -  the  "Numbers  Game"  -  has  been 
shown  to  be  a  useful  tool  for  studying  querying-reasoning  behavior 
and  developing  methods  of  self-instruction  in  this  behavior. 

2.  Querying-te&soning  behavior,  as  represented  by  the 
experimental  task,  can  be  beneficially  modified  with  practice  and 
training. 
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C.  The  Phylogenic  Concept  of  Program  Order 


1.  A  conceptual  approach  to  ordering  content  in  programed 
instruction  has  been  developed.  This  involves  a  "method  of  discovery.  " 
Z.  An  experimental  model  for  studying  this  approach  has  been 
designed  and  developed. 
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Appendix  I 


Symbology  for  Representing  Logical  Relationships 
Between  Task  Components 

Causation  or  sequence  proceeds  from  left  to  right,  generally  in  the 
following  sequence: 


□ - O - □ - □ - 0-^etc- 

display  man  control  display  man 

(action) 

If.only  one  man  performs  the  entire  task,  the  man  symbol  may  be  eliminated. 
In  many  tasks  a  variety  of  control  actions  will  be  accomplished  between  dis¬ 
play  changes,  or  several  displays  may  be  used  to  specify  our  action.  In  such 
cases  the  straight  lines  which  indicate  the  logical  relationships  will  have  to 
branch  accordingly.  The  following  tabulation  of  branchings  are  suggested  for 
use  with  the  present  set  of  symbols. 


A.  The  "and"  branches: 

1.  Unordered  do  all  in  any  order 


observe  all  in  any  order 


action  leads  to  several 
displays 
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0^0 

No 


2.  Sequential  (ordered) 


do  direct  line,  then 
next  branch 


observe  direct  line, 
then  next  branch 


□P 


CH 


O 


3.  Simultaneous  (ordered) 

do  coordinated  action 


observe  in  a 
coordinated  fashion 


B.  The  "or"  branches: 

1 .  Contingent  (indicated  by  an  open  arrow) 


do  the  action  specified 
by  the  previous  display 
or  decision  path 


observe  the  display 
specified  by  the  previous 
decision  path  or  display 


action  leads  to  display 
according  to  previous 
decision  path  or  display 
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2.  Whims  (indicated  by  closed  arrow) 


3. 


do  any  one 


observe  any  one 


action  randomly  leads 
to  any  one 


Combination  contingent  and  whim 


a 


do  top  branch  if  condition  5; 
otherwise  select  any 
action 


ob serve  top  one  if 
condition  5;  other¬ 
wise  observe 
any  display 


action  leads  to  top  display 
if  condition  5;  other¬ 
wise  randomly  leads 
to  any  one 
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Various  combinations  of  the  above  branchings  are  permissible. 

For  example,  several  response  paths  may  converge  at  the  same  decision 
point.  In  the  example  below  any  of  the  four  actions  at  the  right  represent 
equally  acceptable  strategies  when  the  decision  point  is  reached  by  paths 
A  or  B.  However,  if  path  C  is  followed,  do  D  if  condition  4  or  6,  do  E 
if  condition  2,  do  F  if  condition  7. 
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Appendix  II 


Logic  Diagrams 

Drawing 

Title 

A-201-1 

A-201-2 

A-201-3 

A-201-4 

A-201-5 

A-201-6 

A-201-7 

A-201-8 

A-201-9 

A-201-10 

A-201-11 

A-201-12 

A-201-13 

A-201-14 

A-201-15 

A-201-16 

Minor  Sequence  1 

Minor  Sequence  2 

Minor  Sequence  3 

Minor  Sequence  4 

Minor  Sequence  5 

Minor  Sequence  6 

Minor  Sequence  7 

Minor  Sequence  8 

Minor  Sequence  9 

Minor  Sequence  10 

Minor  Sequence  11 

Minor  Sequence  12 

Minor  Sequence  13 

Minor  Sequence  14 

Minor  Sequence  15 

Minor  Sequence  16 

B-201-1 

B-201-2 

B-201-3 

Typical  Display  Circuit 

Self  Latching  Relay  and  Prime  Pair 
Major  Sequence  Control 

C-201-1 

C-201-2 

Major  Sequence 

Sequence  Summary 
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Appendix  III 


A  summary  of  hypotheses  and  lemmas  to  be  used  as  a  guide 
in  the  development  of  the  phylogenic  method 

1.  The  creator  of  a  system  knows  that  system  best,  both  functionally  and 
mechanically,  since  each  component  added  was  born  of  his  real  and 
current  need  for  its  specific  function. 

2.  The  actual  evolutionary  steps  through  which  a  system  has  developed  are 
a  good  starting  point  for  a  phylogenic  teaching  program. 

3.  The  information  displayed  to  an  operator,  on  the  basis  of  which  he  must 
make  decisions  to  control  his  actions,  has  a  semantic  content  rather 
than  being  merely  a  description  of  some  current  state  of  the  system. 

4.  The  training  problems  or  environmental  situations  need  not  proceed  in 
a  real  time  or  geometry. 

5.  Each  control  or  display  presented  in  answer  to  a  request  must  provide 
only  one  additional  function  at  that  time. 

6.  Trainees  must  be  taught  to  abstract  the  simplest  possible  verbalization, 
in  the  machine  language,  of  the  function  being  requested. 

7.  At  the  time  of  presentation  a  new  control,  wherever  possible,  should 
offer  to  the  subject  a  means  for  continuous  perturbation  of  the  system 
or  environment. 

8.  Some  requests  by  the  trainee  may  be  irrelevant  or  impossible  to  grant 
or  forehanded;  he  should  be  informed  of  this  at  the  time  of  the  request. 

9.  The  trainee's  role  in  the  phylogenic  teaching  process  must  be  taught  to 
him  initially. 

10.  The  trainee  need  only  request  functions,  not  structures. 

11.  Retraining  of  operators  on  updated  systems  requires  only  the  further 
application  of  a  process  already  learned  by  him. 

12.  In  the  interests  of  a  smooth  and  logical  development,  some  components 
may  have  to  be  offerred  to  the  trainee  which  are  later  destined  to  atrophy 
through  disuse. 
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Appendix  IV 


A  suggested  experimental  model  of  the  phylogenic 
teaching  method. 

A  simple  approach  to  the  building  of  an  experimental  system  for  study 
of  the  phylogenic  method  and  using  a  minimum  of  special  instrumentation 
would  involve  the  manipulation  of  Lissajou  figure  displays  on  an  oscilloscope 
screen.  By  means  of  a  relay  or  stepper -switch  commutation  system  a  num¬ 
ber  of  figures  may  be  presented  in  sequence  and  if  they  are  all  displayed  on 
a  long-persistence  phosphor,  they  will  all  appear  to  exist  simultaneously. 
Each  figure  will  be  controlled  by  a  separate  passive  network  allowing  control 
of  position>  size,  as  over-all  gain  and  in  each  axis  separately,  and  "round¬ 
ness"  (quadrature  of  phase),  and  for  each  there  will  be  a  diode  "breakpoint" 
in  each  axis  which  may  be  placed  anywhere  in  the  figure.  While  the  above 
controls  are  all  of  a  continuous  nature,  toggle  switches  will  be  provided  to 
flip  the  figure  to  a  mirror  image  about  itself  or  about  either  of  the  axes  of 
the  scope.  A  single  sinusoidal  oscillator  source  and  DC  bias  supply  will 
provide  the  input  signals  for  all  networks. 

A  goal  will  be  set  for  the  subject  in  the  form  of  a  pattern  on  a  trans¬ 
parent  overlay  on  the  scope  face  which  he  is  to  imitate  precisely  with  the 
figures  at  his  command.  In  this  first  model  an  experimenter  or  instructor 
will  be  at  hand  to  act  as  interpreter  between  the  subject  and  the  system  and 
to  reveal  controls  as  their  function  is  requested.  Later,  the  experiments 
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Appendix  IV  continued. 


may  be  performed  using  a  digital  computer  generating  patterns  on  an  output 
scope  and  providing  the  subject  with  one  or  more  "light  pens"  with  which  to 
indicate  on  the  pattern  itself  the  perturbation  desired.  Even  in  the  initial 
experiments  more  complexity  in  the  figures  and  the  controlling  functions 
may  be  achieved  by  using  an  analogue  computer  with  operational  amplifiers 
to  generate  the  figures. 

The  teaching  process  might  start  with  the  presentation  of  a  single 
elliptical  figure  of  the  correct  size,  shape,  and  orientation  to  match  the 
first  overlay  but  out  of  position  in  one  direction,  the  next  overlay  would 
require  displacement  in  both  directions,  the  next  a  change  in  orientation 
or  size,  the  next  the  insertion  of  a  breakpoint,  and  so  on.  Eventually  a 
second  figure  vso  uld  be  required,  and  if  all  of  the  manipulations  of  a  single 
one  have  already  been  tought,  then  when  the  second  is  requested,  its  entire 
control  panel  may  be  revealed  at  once.  Shortcut  functions  such  as  a  change 
of  gain  over  the  whole  scope  face  might  be  requested  and  may  be  offered. 

Two  other  figure  variations  which  could  be  included  easily  are  a 
brightening  (or  z-axis)  control  and  a  broadening  of  the  figure's  lines  in 
either  direction  achieved  by  the  insertion  of  low -gain,  high-frequency  sig¬ 
nals  from  another  oscillator  source. 
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Appendix  IV  continued 


At  any  point  during  training  the  subject's  current  proficiency  may 
be  tested  by  presenting  him  with  a  new  and  different  overlay  which  may 
be  matched  using  only  the  controls  he  has  acquired  already  and  measur¬ 
ing  the  time  necessary  to  achieve  a  duplicate. 
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